An Improved Generative Adversarial Network for Micro-expressions based on Multi-label Learning from Action Units

Micro-expression is a spontaneous facial expression with short duration, low intensity and facial partial action units. Micro-expression recognition plays an important role in psychological diagnosis, lie detection, and security systems. However, even the state-of-the-art recognition models suffer from the lack of micro-expression samples. In order to augment the training data, we propose a new method — an improved Generative adversarial network (GAN) for micro-expressions based on multi-label learning from action units. In the proposed model, action units (AUs) are added to GAN in the form of multi-labels. The designed loss function ensures to generate high quality images. The designed loss function for video can obtain a smooth action video with a trajectory of the AU. The designed loss function of optical flow guarantees low intensity for micro-expression generation and high similarity with the original training micro-expression samples. Moreover, the micro-expression recognition accuracy can be improved via adding the generated samples to the training data set. The experimental results on the two benchmark databases including CASME II and MMEW demonstrate the superiority of the proposed method over other state-of-the-art micro-expression recognition methods.