Adaptive Graph Convolutional Network with Prior Knowledge for Action Recognition

Skeleton-based action recognition has been paid more and more attention in recent years. Previous researches mainly depend on CNNs or RNNs to capture dependencies among sequences. Recently, graph convolution networks are widely used due to its extraordinary ability to exploit node relationships. We propose a new GCN-based model named PK-GCN which utilizes prior knowledge to design learnable node connections. It can be proved that models can learn adaptive connections by itself, because the node connections can be learned with random initialization. The prior knowledge can be used to design node connections by selecting prominent pairs of joints in actions. By combining the proposed methods above, PK-GCN achieves the best performance in ablation study. Compared with other single-stream GCN-based models, PK-GCN on two large-scale datasets NTU-RGB+D and Kinetics achieves state-of-the-art results.