Multi-Sensor Information Integration and Automatic Understanding

Abstract : The purpose of this program is to address the development of algorithms for adaptive processing of multi-sensor data, employing feedback to optimize the linkage between observed data and sensor control. The envisioned multi-modal adaptive system is applicable for intelligence, surveillance, and reconnaissance (ISR) in general environments, addressing base and port security, as well as urban and suburban sensing during wartime and peacekeeping operations. Of significant importance for current and anticipated DoD activities, the ISR system is designed to detect asymmetric threats, with the goal of recognizing unusual behavior or activities. Technologies and systems developed under this effort will be designed for semi-automated scene awareness, with the objective of recognizing behavior that appears atypical (e.g. atypical object motion, and dynamic characteristics of people and vehicles). Leveraging their previously developed technology, SIG is developing second-generation methods to adaptively learn the statistics of dynamic object behavior in video, while focusing on defining system requirements for sensor deployment by using field data (vs. highly controlled indoor data). Over the course of the past 2 months, significant progress has been made towards adding the final features necessary for the video tracking system. The code has successfully transitioned into its final format in C for efficient implementation, allowing one to perform significant optimizations and achieve very efficient running times, on the order of 35 frames per second. The authors also have added important capabilities for object tracking across multiple cameras, object classification (allowing behavior analysis to be conditioned on the type of object observed), and virtual pan/tilt/zoom capabilities.