Relation-Guided Actor Attention for Group Activity Recognition

Group activity recognition has received significant interest due to its widely practical applications in sports analysis, intelligent surveillance and abnormal behavior detection. In a complex multi-person scenario, only a few key actors participate in the overall group activity and others may bring irrelevant information for recognition. However, most previous approaches model all the actors’ actions in the scene equivalently. To this end, we propose a relation-guided actor attention (RGAA) module to learn reinforced feature representations for effective group activity recognition. First, a location-aware relation module (LARM) is designed to explore the relation among pairwise actors’ features in which appearance and position information are both considered. We propose to stack all the pairwise relation features and the features themselves of an actor to learn actor attention which determines the importance degree from local and global information. Extensive experiments on two publicly benchmarks demonstrate the effectiveness of our method and the state-of-the-art performance is achieved.