Don’t Blindly Trust Your CNN: Towards Competency-Aware Object Detection by Evaluating Novelty in Open-Ended Environments

Real-world missions require robots to detect objects in complex and changing environments. While deep learning methods for object detection are able to achieve a high level of performance, they can be unreliable when operating in environments that deviate from training conditions. However, by applying novelty detection techniques, we aim to build an architecture aware of when it cannot make reliable classifications, as well as identifying novel features/data. In this work, we have proposed and evaluated a system that assesses the competence of trained Convolutional Neural Networks (CNNs). This is achieved using three complementary introspection methods: (1) a Convolutional Variational Auto-Encoder (VAE), (2) a latent space Density-adjusted Distance Measure (DDM), and (3) a Spearman’s Rank Correlation (SRC) based approach. Finally these approaches are combined through a weighted sum, with weightings derived by maximising the correct attribution of novelty in an adversarial ‘meta-game’. Our experiments were conducted on real-world data from three datasets spread across two different domains: a planetary and an industrial setting. Results show that the proposed introspection methods are able to detect misclassifications and unknown classes indicative of novel features/data in both domains with up to 67% precision. Meanwhile classification results were either maintained or improved as a result.