Human-Aware Coarse-to-Fine Online Action Detection

In this work, we propose a two-stage framework to efficiently and effectively detect actions on-the-fly. An action location network (ALN) is developed in the first stage to judge whether the current frame is action-related, while the second stage involves an action classification network (ACN) to further identify the action category. In this way, irrelevant negative frames are quickly discarded and actions are detected as early as they occur. Moreover, we highlight human areas at both the stages by respectively incorporating a human detector and a human mask layer. As a result, more accurate spatial-temporal windows of actions are detected, based on which more robust features are extracted for classification. Experimental results on two popular benchmarks demonstrate the superior performance of the proposed approach.