CNN-based driving maneuver classification using multi-sliding window fusion

Abstract Driving behavior classification has received increasing attention in recent years, where driving maneuver classification plays an important role. The first step of building a driving maneuver classification system is to segment maneuvers, which is often realized by using a single sliding window in previous work. However, different types of driving maneuvers often have different maneuver duration. It is difficult to segment those maneuvers using only a single fixed-sized window. In this paper, we present a CNN-based method to classify driving maneuvers using multi-sliding window fusion. First, multi-sliding windows of both short and longer sizes are used for constructing a robust feature set. Then, CNN-based mid-fusion is used for classifying driving maneuvers. To evaluate the proposed approach, a public dataset named UAH-DriveSet with six drivers driving on the highway is used. Six driving maneuvers were labeled: lane keeping, braking, turning, acceleration, right lane change, and left lane change. The experimental results show that our proposed CNN-based driving maneuver classification can achieve a macro F1-score of 58.22% using single-window and early-fusion. Comparing four different fusion methods, All fusion achieves the best performance. With multi-sliding window fusion and mid-fusion based CNN, the highest macro F1-score can be up to 80.25%, which is higher than early- and late- fusion. In addition, the F1-score of CNN-based method is higher than both k-NN and RF-based methods. Finally, we verify the importance of label information for driving maneuver classification, and the highest macro F1-score is 87.67% with an assigned duration of 4s.