Unsupervised Anomaly Detection for Traffic Surveillance Based on Background Modeling

Most state-of-the-art anomaly detection methods are specific to detecting anomaly for pedestrians and cannot work without adequate normal training videos. Recently, there is a growing demand for detecting anomalous vehicles in traffic surveillance videos. However, the biggest challenge in this task is the lack of labeled datasets for training supervised models. By examining the resemblances of anomalous vehicles, we find it reasonable to label a vehicle as anomaly if it stays still in the video for a relatively long time. Utilizing this property, in this paper we introduce a novel unsupervised anomaly detection method for traffic surveilliance based on background modeling, which shows great potentials in handling heterogeneous scenes as well as extremely low resolution videos recordings without the dependence on labeled data. In the proposed system, we first employ background modeling using MOG2 to remove the moving vehicles as foreground while keeping the stopped vehicles as part of the background. Then we use Faster R-CNN to detect vehicles in the extracted background and decide if they are new anomalies under certain conditions. All information is updated on a frame basis until the end of the video which contains the final results. In this way, we make full use of the characteristics that abnormal vehicles stay in the scene for a relatively long time and reduce the difficulty of vehicle anomaly detection. Eventually, we can detect almost every anomaly in the NVIDIA AI CITY CHALLENGE track-2 dataset except for several extremely complex cases with a 81.08% F1-score and 10.2369 RMSE.