Looking ahead: Joint small group detection and tracking in crowd scenes

Abstract Small group detection and tracking in crowd scenes are basis for high level crowd analysis tasks. However, it suffers from the ambiguities in generating proper groups and in handling dynamic changes of group configurations. In this paper, we propose a novel delay decision-making based method for addressing the above problems, motivated by the idea that these ambiguities can be solved using rich temporal context. Specifically, given individual detections, small group hypotheses are generated. Then candidate group hypotheses across consecutive frames and their potential associations are built in a tree. By seeking for the best non-conflicting subset from the hypothesis tree, small groups are determined and simultaneously their trajectories are got. So this framework is called joint detection and tracking. This joint framework reduces the ambiguities in small group decision and tracking by looking ahead for several frames. However, it results in the unmanageable solution space because the number of track hypotheses grows exponentially over time. To solve this problem, effective pruning strategies are developed, which can keep the solution space manageable and also improve the credibility of small groups. Experiments on public datasets demonstrate the effectiveness of our method. The method achieves the state-of-the-art performance even in noisy crowd scenes.