Two-Stream Convolutional Neural Network for Skeleton-Based Action Recognition

Skeleton-based action recognition has attracted much attention while the skeleton contains important information related to action. How to efficiently use the temporal and spatial information in skeleton data is still an open problem. We propose in this paper a two-stream convolutional neural network which combines joints information with bone information to recognize actions. More specifically, the input of one stream is the joints position information (point information) and the other is the bone vector information (direction and length information) which is more informative in local relative change. Every stream also integrates motion information which represent temporal difference of inputs. Besides, we enhance our network using asymmetric convolution blocks (ACBlock). ACBlock can capture features efficiently and reduce the negative effects of rotation and deformation which contributes to feature extraction. Experiments on NTU-RGB+D datasets demonstrate the advantage of the proposed model on effectiveness of the novel model to recognize the action.