Real-time action detection and temporal segmentation in continuous video

ABSTRACTTemporal segmentation of actions has been under intensive focus in the field of computer vision for a prolonged period. The present study proposed a template-based framework to resolve the issues concerning timeliness and real-time performance in the temporal segmentation in a continuous video. A complete action can be detected, based on the previous frames, and the action can be segmented immediately without waiting for the follow-up frames. Herein, characteristic frames are selected by a martingale-based method, followed by the formation of the corresponding motion history through backtracking along the characteristic frames, and the final segmentation is determined according to the recognition model trained by the extreme learning machine. In the experiment on the IXMAS database, the average rate of the detection of action reached 91%, and the accuracy in the frame level reached 83.5%. In the experiment on the 3D skeleton data based on Kinect, the detection rate reached 94%.