Multiple Kernel Learning and Optical Flow for Action Recognition in RGB-D Video

Recognizing human action is valuable for many real world applications such as video surveillance, human computer interaction, smart home and gaming. In this paper, we present a method of action recognition based on hypothesizing that the classification of action can be boosted by motion information using optical flow. Emergence of automatic RGBD video analysis, we propose fusing optical flow is extracted from both RGB and depth channels for action representation. Firstly, we extract optical flow from RGB and depth data. Secondly, motion descriptor with spatial pyramid is computed from histogram of optical flow of RGB and depth. Then, feature pooling technique is used in order to accumulate RGB and depth feature into set of feature vectors for each action. Finally, we use the Multiple Kernel Learning (MKL) technique at the kernel level for action classification from RGB and depth feature pooling. To demonstrate generalizability, our proposed method has been systematically evaluated on two benchmark datasets shown to be more effective and accurate for action recognition compared to the previous work. We obtain overall accuracies of: 97.5 % and 92.8 % with our proposed method on the 3D Action Pairs and MSR-Daily Activity 3D dataset, respectively.