Reducing Network Agnostophobia

Agnostophobia, the fear of the unknown, can be experienced by deep learning engineers while applying their networks to real-world applications. Unfortunately, network behavior is not well defined for inputs far from its training. In an uncontrolled environment, networks face many instances which are not of interest to them and have to be rejected in order to avoid a false positive. This problem has previously been tackled by researchers by either a) thresholding softmax, which by construction must return one of the known classes, or b) using an additional background or garbage class. In this paper, we show that both of these approaches help, but are generally insufficient when previously unseen classes are encountered. We introduce a new evaluation metric which focuses on comparing the performance of multiple approaches in scenarios where unknowns are encountered. Our major contributions are our simple yet effective Entropic Openset, and Objectosphere losses, which similar to the current approaches train with negative samples. However, these novel losses are designed to maximize entropy for unknown inputs while also increasing separation in deep feature magnitude between known and unknown classes. Experiments on MNIST and CIFAR-10 show that our novel loss is significantly better at dealing with unknown inputs from datasets such as letters, Not MNIST, Devanagari, and SVHN.