Summarizing surveillance videos with local-patch-learning-based abnormality detection, blob sequence optimization, and type-based synopsis

Abstract In this paper, we propose a new approach to detect abnormal activities in surveillance videos and create suitable summary videos accordingly. The proposed approach first introduces a patch-based method to automatically model normal activity patterns and key regions in a scene. In this way, abnormal activities can be effectively detected and classified from the modeled normal patterns and key regions. Then, a blob sequence optimization process is proposed which integrates spatial, temporal, size, and motion correlation among objects to extract suitable foreground blob sequences for abnormal objects. With this process, blob extraction errors due to occlusion or background interference can be effectively avoided. Finally, we also propose an abnormality-type-based method which creates short-period summary videos from long-period input surveillance videos by properly arranging abnormal blob sequences according to their activity types. Experimental results show that our proposed approach can effectively create satisfying summary videos from input surveillance videos.