Action recognition in surveillance videos using semantic web rules

In this paper an approach to detect high level events using Semantic Web Rules (SWRL), will be presented. This approach combines middle-level events and information about actors and actions, extracted from a Visual Analysis module, with a semantic rules inference system to detect meaningful high level crime scenarios. The middle-level events and the spatial and temporal information is indexed in an optimized semantic data-store, where rules for detecting events are manually defined using SWRL. When these rules are applied to the indexed information, high level events can be detected. Early tests of the system successfully detect fights, pickpocketing, thefts and more general “suspicious events”. The work needed to perform this process in CCTV videos in an automated and unattended fashion has been challenging in terms of aggregation of data and optimisation of the different subsystems involved in the process. Specially to make results available in a reasonable time to apply these techniques in a production environment in police stations.