Wearable Facial Action Unit Classification from Near-field Infrared Eye Images using Deformable Models

Developments in head-mounted displays and wearable eyewear devices provide an opportunity to recognize expressions from near-field images, however this is not an area that has received significant research attention to date. In this study, we explore the identification of seven basic upper facial action units by analyzing just part of the face, i.e. infrared eye image, and applying a deformable eye model to obtain detailed information from the eye region. Based on this eye model, a novel feature extraction method is proposed that is a hybrid of geometric and appearance features extracted from around the edge of the eye. Evaluation on a novel database of near-field infrared facial expressions from 16 participants shows that 7-class upper face action unit classification can achieve around 78.8% accuracy using the proposed method, which is promising for wearable applications of automatic facial expression analysis.