Domain Discrepancy Elimination and Mean Face Representation Learning for NIR-VIS Face Recognition

Due to its potential application in criminal cases, security systems and multimedia information retrieval, Near InfraRed (NIR) to VISible (VIS) face recognition has attracted increasing research attention in the field of computer vision. However, it is still a challenging task because of the large intra-class variations including spectrum, occlusion, lighting, blurry, expression and pose. To address the above problem, we propose a novel Domain discrepancy Elimination and Mean face Representation learning (DEMR) for NIR-VIS face recognition. The DEMR consists of two key components comprising Class-wise Domain Discrepancy Elimination (CDDE) and Cross-modal Mean Face Alignment (CMFA). Specifically, two-branch modality-specific networks are designed to extract features for VIS images and NIR images, respectively. Considering that distribution variations of cross-modal images will decrease recognition performance, we present CDDE to eliminate modality gap by narrowing distribution differences of VIS images and NIR images in a category-by-category manner. Moreover, to reduce the intra-class discrepancies and obtain compact feature representation, the CMFA is designed to achieve representation alignment between cross-domain images and VIS prototypes (i.e., VIS mean face representations), through optimizing a quadruplet constraint. Extensive experiments on multiple challenging NIR-VIS databases validate that our DEMR is effective for cross-modal face recognition task.