Traffic light recognition exploiting map and localization at every stage

Abstract Traffic light recognition is being intensively researched for the purpose of reducing traffic accidents at intersections and realizing autonomous driving. However, conventional vision-based approaches have several limitations due to full image scanning, always-on operation, various different types of traffic lights, and complex driving environments. In particular, it might be impossible to recognize a relevant traffic light among multiple traffic lights at multiple intersections. To overcome such limitations, we propose an effective architecture that integrates a vision system with an accurate positioning system and an extended digital map. The recognition process is divided into four stages and we suggest an extended methodology for each stage. These stages are: ROI generation, detection, classification, and tracking. The 3D positions of traffic lights and slope information obtained from an extended digital map enable ROIs to be generated accurately, even on slanted roads, while independent design and implementation of individual recognition modules for detection and classification allow for selection according to the type of traffic light face. Such a modular architecture gives the system simplicity, flexibility, and maintainable algorithms. In addition, adaptive tracking that exploits the distance to traffic lights allows for seamless state estimation through smooth data association when measurements change from long to short ranges. Evaluation of the proposed system occurred at six test sites and utilized two different types of traffic lights, seven states, sloped roads, and various environmental complexities. The experimental results show that the proposed system can recognize traffic lights with 98.68% precision, 92.73% recall, and 95.52% accuracy in the 10.02–81.21 m range.