Deep-Learning and Depth-Map Based Approach for Detection and 3-D Localization of Small Traffic Signs

The three-dimensional (3-D) geographic locations of street furniture, such as traffic signs, comprise the basic content of 3-D city construction, and such information is indispensable for periodic statistics for road management and maintenance. This article presents a novel solution for acquiring 3-D information on small traffic signs based on mobile mapping system (MMS) data. First, a lightweight backbone network called VGG-L under an optimized faster region-based convolutional neural network detection framework is proposed for the detection of small traffic signs. An urban traffic sign detection (UTSD) dataset is created based on panoramic images obtained from test areas. Detection results from the UTSD dataset show that VGG-L outperforms other popular networks and achieves a mean average precision of 75.4%, which is 4.2%–14.8% higher than those of VGG16, MobileNet, ResNet, and YOLOv3. Second, a novel depth-map-based 3-D spatial geolocation method is proposed for obtaining the 3-D geographic locations of the objects. Then, a center-based method is proposed to automatically extract the final 3-D vector of the target. Experimental results illustrate that the proposed method performs 3-D positioning and vectorization of the milestones and circular and triangular traffic signs, accurately and effectively, achieving greater than 86% recall and precision for the three types of targets in the test areas. The experiments demonstrate that the overall 3-D information acquisition scheme is feasible and has great application potential.