On Decomposing an Unseen 3D Face into Neutral Face and Expression Deformations

This paper presents a technique for decomposing an unseen 3D face under any facial expression into an estimated 3D neutral face and expression deformations (the shape residue between the non-neutral and the estimated neutral 3D face). We show that this decomposition gives a robust facial expression classification and improves the accuracy of an off-the-shelf 3D face recognition system. The proposed decomposition system is a multistage data-driven process in which training expression residues and neutral faces reciprocally guide the decomposition of the 3D face. A plausible decomposition was achieved. The shapes and the normals of the expression residue are used for expression classification while the neutral face estimates are used for expression robust face recognition. Experiments were performed on a large number of non-neutral scans and significant expression classification rates were achieved. Moreover, 6% increase in face recognition rate was achieved for probes with severe facial expressions.