Action Recognition Using Multi-Temporal DMMs Based on Adaptive Vague Division

Depth motion maps (DMMs) extracted from the whole video sequence has inherent shortages. This paper proposes a novel effective method based on depth video sequences for action recognition. First, each depth sequence is divided into several sub-sequences referring to the accumulative motion energy of the sequence. Due to the different density of motion energy in action sequences, we obtain sub-sequences in different lengths by equally dividing the energy. In order to utilize the vague motion energy, we use a parameter α which controls the percent between the sub-sequences and their adjacent sequences to construct sequences. These sequences are denoted as VME-sequences. Second, we calculate Multi-Temporal DMMs of each VME-sequence which is projected to three views (front, side and top) to adapt time and speed variation. Then we use local binary patterns (LBPs) for each projected views and Fisher kernel is applied to encode the patch descriptors which result to a compact feature representation. For classification, we apply kernel based extreme learning machine (KELM). Experiments on three general datasets: MSR Action Pairs, MSR Gesture 3D and MSR Action3D datasets have shown better result than most existing methods.