Multiple Target Tracking with a Steerable Airborne Video Sensor

Tracking multiple surface targets with a single steerable airborne video sensor is accomplished by several interrelated functions: (i) image registration for camera motion compensation and accurate image-to-ground mapping, (ii) video processing for object detection and feature extraction, (iii) target tracking for detection association and track creation and maintenance, (iv) signature extraction and exploitation, and (v) sensor resource management for the generation of sensor steering commands. The first function is often overlooked, but has a significant impact in the performance of the overall system. A rudimentary registration can be achieved by using the platform location and attitude as well as the sensor orientation and field of view information, but the accuracy of this registration is typically poor due to inertial navigation system errors, particularly in small unmanned aerial vehicles with cost and hardware limitations. Successful registration of successive frames enables the use of multiple frame video processing for improved object detection and provides stable image-to-ground mapping for improved data association by the tracker. In systems with a steerable sensor that slews back and forth to track more than one target simultaneously, the image registration module creates and maintains multiple mosaics corresponding to the different tracking areas. In this paper we discuss primarily the image registration module and the system of coordinate frames that is maintained to improve data association and tracking.