Vessel classification for naval operations

An important surveillance task during naval military operations is early risk assessment of vessels. The potential risk that a vessel poses will depend on the vessel type, and vessel classification is therefore a basic technique in risk assessment. Although automatic identification by AIS is widely available, the AIS transponders can potentially be spoofed or disabled to prevent identification. A possible complementary approach is the use of automatic classification based on camera imagery. The dominant approach for visual object classification is the use of deep neural networks (DNNs), which has shown to give unparalleled performance when sufficiently large annotated training data sets are available. However, within the scenario of naval operations there are several challenges that need to be addressed. First, the number and types of classes should be defined in such a way that they are relevant for risk assessment while allowing sufficiently large training sets per class type. Second, early risk assessment in real-life conditions is vital and vessel type classification should work on long range target imagery having low-resolution and being potentially degraded. In this paper, we investigate the performance of DNNs for vessel classification under the aforementioned challenges. We evaluate different class groupings for the MARVEL vessel data set, both from an accuracy perspective and the relevancy for risk assessment. Furthermore, we investigate the impact of real-life conditions on classification by manually downsizing and reducing contrast of the MARVEL imagery, as well as evaluating on EO/IR recordings from Rotterdam harbor which has been collected for several weeks under varying weather conditions.