Autoencoders Covering Space as a Life-Long Classifier.

A life-long classifier that learns incrementally has many challenges such as concept drift, when the class changes in time, and catastrophic forgetting when the earlier learned knowledge is lost. Many successful connectionist solutions are based on an idea that new data are learned only in a part of a network that is relevant to the new data. We leverage this idea and propose a novel method for learning an ensemble of specialized autoencoders. We interpret autoencoders as manifolds that can be trained to contain or exclude given points from the input space. This manifold manipulation allows us to implement a classifier that can suppress catastrophic forgetting and adapt to concept drift. The proposed algorithm is evaluated on an incremental version of the XOR problem and on an incremental version of the MNIST classification where we achieved 0.9 accuracy which is a significant improvement over the previously published results.