Deep domain adaptation with manifold aligned label transfer

We propose a novel deep learning domain adaptation method that performs transductive learning from the source domain to the target domain based on cluster matching between the source and target features. The proposed method combines Adaptive Batch Normalization and Locality Preserving Projection-based subspace alignment on deep features to produce a common feature space for label transfer. Adaptive Batch Normalization automatically conditions the features from the source/target domain by normalizing the activations in each layer of our network. Following Manifold Subspace Alignment, we cluster the data in each domain using Gaussian Mixture Model clustering in feature space. The clusters are matched between domains to transfer labels from the closest source cluster to each target cluster. The transfer labels are compared to the network prediction, and the samples with consistent labels are used to adapt the network on the target domain. The proposed manifold-guided label transfer method produces state-of-the-art results for deep adaptation on digit recognition datasets. Furthermore, we perform domain adaptation on remote sensing datasets.