Real-time 3D face reconstruction from one single image by displacement mapping

In this paper, we present a fast and robust method to reconstruct a plausible three-dimension (3D) face from one single frontal face image. In training phase, we classify the faces into several groups based on the facial structures and propose to learn a mapping, known as the displacement mapping (DM) in this paper, for each group. DM relates two displacements: One displacements, denoted as 2D displacements, represent the differences between the positions of feature points on the 2D training faces and those on the reference 2D face that has been pre-defined for the corresponding group; another displacements, denoted as 3D displacements, are the differences between the positions of vertices on the reconstructed 3D face and those on the reference 3D face that is also pre-defined. During the reconstruction phase, we first classify the input face as one of the groups and calculate the 2D displacements. Then we take advantage of the 2D displacements and the learned DM to estimate the 3D displacements. Subsequently, 3D displacements can be used to obtain the precise 3D face by shifting the 3D reference face. Experiments on Basel face model (BFM) database as well as some real-world 2D face images demonstrate the effectiveness and efficiency of the proposed method, in comparison with some state-of-arts methods.