Detecting and Tracking Animals in Underwater Video Using a Neuromorphic Saliency-based Attention System

The Monterey Bay Aquarium Research Institute (MBARI) uses highresolution video equipment on remotely operated vehicles to obtain quantitative data on the distribution and abundance of oceanic animals. Highquality video data supplants the traditional tow net approach of assessing the kinds and numbers of animals in the oceanic water column. Video camerabased quantitative video transects (QVT) are taken through the ocean midwater, from 50 m to 4000 m, and provide high-resolution data at the scale of the individual animals and their natural aggregation patterns. However, the current manual method of analyzing QVT video is labor intensive and tedious. We present an automated system for detecting marine organisms visible in the videos. Video frames are processed with a neuromorphic selective attention algorithm. The candidate locations are tracked across video frames using linear Kalman filters. If objects can be tracked successfully over several frames, they are labeled as potentially "interesting". Based on low-level properties, objects are classified as interesting and marked in the video frames. The system enhances the productivity of human video annotators and/or cues a subsequent object classification module by marking candidate objects.