Multi-tasking deep convolutional network architecture design for extracting nonverbal communicative information from a face

Abstract Facial expressions convey not only emotions but also communicative information. Therefore, facial expressions should be analysed to understand communication. The objective of this study is to develop an automatic facial expression analysis system for extracting nonverbal communicative information. This study focuses on specific communicative information: emotions expressed through facial movements and the direction of the expressions. We propose a multi-tasking deep convolutional network (DCN) to classify facial expressions, detect the facial regions, and estimate face angles. We reformulate facial region detection and face angle estimation as regression problems and add task-specific output layers in the DCN’s architecture. Experimental results show that the proposed method performs all tasks accurately. In this study, we show the feasibility of the multi-tasking DCN for extracting nonverbal communicative information from a human face.