Deep-Learning Based Facial Expression Recognition System Evaluated on Three Spontaneous Databases

Feature extraction and selection are significant operations to improve the recognition accuracy of facial expression systems. The distribution of geometric features and their quantity plays a decisive role in the quality of the process of image matching, particularly for some databases which have more challenges in terms of system accuracy. In this paper, we exploit a robust system to mitigate these challenges as this is essential for real-time applications. We concentrate on geometric feature extraction automatically from raw data with one of the most attractive methods for classification in the field of neural networks namely deep learning. Our improved system consists of the following: solving the misalignment problem of the training images, lower complexity for geometric feature extraction, and finally, auto-encoder deep learning. The performance of the image-based expression recognition is evaluated for the first time on three spontaneous databases with different challenges with geometric and appearance based features for comparison. The three spontaneous databases are the Video Database of Moving Faces and People (VDMFP), MMI facial expression database and Belfast Induced Natural Emotion Database (BINED) each having different challenges in terms of system accuracy. Deep learning with a high-level feature representation, clearly outperforms state-of-the-art techniques.