Enhancing Image Representations for Occluded Face Recognition via Reference Conditioned Low-Rank projection

Deep learning in face recognition is widely explored in recent times due to its ability to produce state-of-the-art results and availability of large public datasets. While recent deep learning approaches involving margin loss based image representations produce 99% accuracy across benchmarks, none of these studies focus explicitly on occluded face verification. Further, in real world scenarios, there is a need for efficient methods that cater to the cases of occlusion of faces with hats, scarves, goggle or sometimes exaggerated facial expression. Moreover, with face verification gathering traction in mainstream real-time embedded applications of surveillance, the proposed approaches need to be highly accurate. In this paper, we revisit the same through a large-scale study involving multiple synthetically created goggle-occluded face datasets using multiple state-of-the-art face representations. Through this study, we identify that occlusion in faces results in non-isotropic face representations in feature space which results in a drop in performance. Therefore, we propose an approach to enhance existing face representations by learning reference conditioned Low-Rank projections (RCLP), which can create isotropic representations thereby improving face recognition. We benchmark the developed approach over synthetically goggled versions of LFW, CFP-FP, ATT, FEI, Georgia Tech and Essex University face databases with representations from ResNet-ArcFace, VGGFace, MobilefaceNet-ArcFace LightCNN resulting in a total of 100 + experiments where we achieve improvements in the accuracy-rate across all with a maximum of 4% on FEI dataset. Finally, to validate the approach in a realistic scenario, we additionally present results over our internal face verification dataset of 1k images and confirm that the proposed approach only shows positive results without degrading existing baseline performance.