Hard Class Rectification for Domain Adaptation

Abstract Domain adaptation (DA) aims to transfer knowledge from a label-rich and related domain (source domain) to a label-scare domain (target domain). Pseudo-labeling has recently been widely explored and used in DA. However, this line of research is still confined to the inaccuracy of pseudo labels. In this paper, we explore the imbalance issue of performance among classes in-depth and observe that the worse performances among all classes are likely to further deteriorate in the pseudo-labeling, which not only harms the overall transfer performance but also restricts the application of DA. In this paper, we propose a novel framework, called Hard Class Rectification Pseudo-labeling (HCRPL), to alleviate this problem from two aspects. First, we propose a simple yet effective scheme, named Adaptive Prediction Calibration (APC), to calibrate predictions of target samples. Then, we further consider the predictions of calibrated ones, especially for those belonging to the hard classes, which are vulnerable to perturbations. To prevent these samples to be misclassified easily, we introduce Temporal-Ensembling (TE) and Self-Ensembling (SE) to obtain consistent predictions. The proposed method is evaluated on both unsupervised domain adaptation (UDA) and semi-supervised domain adaptation (SSDA). Experimental results on several real-world cross-domain benchmarks, including ImageCLEF, Office-31, Office+Caltech, and Office-Home, substantiate the superiority of the proposed method.