Discriminative regularization of the latent manifold of variational auto-encoders

Abstract We present an approach on training classifiers or regressors using the latent embedding of variational auto-encoders (VAE), an unsupervised deep learning method, as features. Usually VAEs are trained using unlabeled data and independently from the classifier, whereas we investigate and analyze the performance of a classifier or regressor that is trained jointly with the variational deep network. We found that models trained this way can improve the embedding s.t. to increase classification performance, and also can be used for semi-supervised learning, building up the information extracting latent representation in an incremental fashion. The model was tested on two widely known computer vision benchmarks, and its generalization power was evaluated on an independent dataset. Additionally, generally applicable statistical methods are presented for evaluating similarly performing classifiers, and used to quantify the performance increase. The general applicability and ease-of-use of deep learning approaches allows for a wide applicability of the method.