Expression-targeted feature learning for effective facial expression recognition

Abstract In this paper, we propose a novel expression-targeted feature learning (ETFL) method for effective facial expression recognition, which takes advantage of multi-task learning for discriminative feature learning. Specifically, the common features are firstly extracted from the lower layers of CNN. Then, based on the common features, the expression-specific features (ESF) are respectively learned for each facial expression via multi-task learning. In order to enhance the discriminability of ESF, we develop a joint loss (the combination of the center loss and a novel inter-class loss) to explicitly reduce intra-class variations while enlarging inter-class differences. Furthermore, we introduce the sample-sensitive weights and the soft-expression weights to balance the joint loss for better performance. Finally, all ESFs are combined for final classification. ETFL effectively exploits the relationship among all facial expressions, which leads to superiority feature discriminability. Experiments on public facial expression databases demonstrate the effectiveness of ETFL compared with several state-of-the-art methods.