Data-Driven Machine Learning Approach for Human Action Recognition Using Skeleton and Optical Flow

Human action recognition is a very challenging problem due to numerous variations in each body part. In this paper, we propose a method for extracting optical flow information from skeleton data to address the problem of body part movement variation in human action recognition. The additional arm part information was also analyzed how valuable it was. Then, different machine learning methods are applied such as k-Nearest Neighbors (KNNs) and deep learning to recognize human actions on the UTKinect-Action 3D dataset. We then design and train different KNNs models and Deep Convolutional Neural Networks (D-CNNs) on the obtained image and classify them into classes. Different numbers of features from histogram data collection are used to recognize 10 categories of human actions. The best accuracy we obtained is about 88%. The proposed method had improved accuracy to almost 97% with only 5 classes. These features are representative to describe the human action and recognition which does not rely on plenty of training data. Results of experiments show that using deep learning can lead to better classification accuracy.