Visual analysis of socio-cognitive crowd behaviors for surveillance: A survey and categorization of trends and methods

Abstract Monitoring and inferring socio-cognitive behaviors through crowd analysis can help us to understand many processes. Be it people in crowded environments, road traffic or even a flock of fish, situational awareness becomes critical for creating adequate disaster response, providing incident management, exercising control, etc. Recent researches have indicated that crowd modeling is conventionally based on density analysis. However, socio-cognitive behavior studies have demonstrated that crowds often display a wide variety of behaviors that arise spontaneously from the collective motions of unconnected individuals. Therefore, behavior analysis employing physics-based approaches only, thereby neglecting the socio-psychological aspects, may present diverse challenges to accurate inference. This means that by identifying and modeling some of the interacting agents that underpin the evolution of such behaviors, we can deliver contexts that can help in the autonomous analysis of social and antisocial behaviors in crowded environments. This paper discusses these issues from the machine vision perspective. In particular, socio-cognitive models of crowds are linked to low-level mechanisms of crowd modeling and feature extraction. A survey of recent works on crowd behavior analysis is conducted under a proposed behavioral categorization based on the level of the performed analysis and identified behaviors. Finally, discussions and recommendations are provided toward the advancement in the field.