Human emotion recognition based on facial expressions via deep learning on high-resolution images

Detecting human emotion based on facial expression is considered a hard task for the computer vision community because of many challenges such as the difference of face shape from a person to another, difficulty of recognition of dynamic facial features, low quality of digital images, etc. In this paper, we propose a face-sensitive convolutional neural network (FS-CNN) for human emotion recognition. The proposed FS-CNN is used to detect faces on large scale images then analyzing face landmarks to predict expressions for emotion recognition. The FS-CNN is composed form two stages, patch cropping, and convolutional neural networks. The first stage is used to detect faces in high-resolution images and crop the face for further processing. The second stage is a convolutional neural network used to predict facial expression based on landmarks analytics, it was applied on pyramid images to process scale invariance. The proposed FS-CNN was trained and evaluated on the UMD Faces dataset. High performance was achieved with a mean average precision of about 95%.