Characterizing terrain image classification difficulties through reduced-dimension class convex hull analysis

Abstract The classification of natural terrain ahead of an autonomous vehicle can help it make decisions regarding the traversability of various paths, but remains an open research problem. Despite the explosion in popularity of deep learning networks,there is still little work available on informed neural network design procedures for specific tasks such as terrain image classification, save through performance measures. A related problem is understanding features of a dataset that lead to difficulties in separating classes of images from one another. This research proposes an algorithm and accompanying analytical procedure to characterize such image classification difficulties; identifying what makes some images easily distinguishable as their class and what makes others readily confused with other classes. This is achieved by learning reduced-dimensionality representations of the input data, constructing a convex hull of class members in the reduced dimensionality representation, then examining between-class overlap within each space, incrementally increasing the dimensionality until overlap is eliminated. Summarizing the between-class overlap statistics reveals trends and anomalies that can be linked back to visual features, characteristic of the original data. Case studies are presented of insights identified through selected example analyses: relative intensities of terrain classes from images taken by Mars rovers, and the impact of color gradients in separating sand from bedrock in color images of terrain. Such insights are discussed as steps toward a more directed approach to designing neural networks for image classification.