Deep Partial Occlusion Facial Expression Recognition via Improved CNN

Facial expression recognition (FER) can indicate a person’s emotion state, that is of great importance in virtual human modelling and communication. However, FER suffers from a partial occlusion problem when applied under an unconstrained environment. In this paper, we propose to use facial expressions with partial occlusion for FER. This differs from the most conventional FER problems which assume that facial images are detected without any occlusion. To this end, by reconstructing the partially occluded facial expression database, we propose a 20-layer “VGG + residual” CNN network based on the improved VGG16 network, and adapt a hybrid feature strategy to parallelize the Gabor filter with the above CNN. We also optimize the components of the model by LMCL and momentum SGD. The results are then combined with a certain weight to get the classification results. The advantages of this method are demonstrated by multiple sets of experiments and cross-database tests.