Face Expression Recognition with Audio-Visual Bi Modal Association

Most automatic expression analysis system attempt to recognize a small set of prototypic expressions, such as happiness, anger, surprise, fear, disgust and sad. We present a novel facial expression recognition framework using audio-visual information analysis. In particular, we design a single good image representation of the image sequence by weighted sum of registered face images where the weights are derived using auditory features. We use a still image based technique for the expression recognition task. We performed experiments using eNTERFACE’05 audio-visual emotional database. The analysis shows that our framework can improve the recognition performance while significantly reducing the computational cost by avoiding redundant or insignificant frame processing by incorporating auditory information. Our analyses show promising results.