Visual Perception Based Situation Analysis of Traffic Scenes for Autonomous Driving Applications

The major challenges for analyzing the situation of traffic scenes include defining proper metrics and achieving computation efficiency. This paper proposes two new situation metrics, a multimodality scene model, and a metrics computing network for traffic scene analysis. The main novelty is threefold. (1) The planning complexity and perception complexity are proposed as the situation metrics of traffic senes. (2) A multimodality model is proposed to describe traffic scenes, which combines the information of the static environment, dynamic objects, and ego-vehicle. (3) A deep neural network (DNN) based computing network is proposed to compute the two situation metrics based on scene models. Using the Nuscenes dataset, a high-level dataset for traffic scene analysis is developed to validate the scene model and the situation metrics computing network. The experiment results show that the proposed scene model is effective for situation analysis and the proposed situation metrics computing network outperforms than traditional CNN methods.