Network Structure for Personalized Face-Pose Estimation Using Incrementally Updated Face-Shape Parameters

This paper proposes a deep learning method for face-pose estimation with an incremental personalization mechanism to update the face-shape parameters. Recent advances in machine learning technology have also led to outstanding performance in applications of computer vision. However, network-based algorithms generally rely on an off-line training process that uses a large dataset, and a trained network (e.g., one for face-pose estimation) usually works in a one-shot manner, i.e., each input image is processed one by one with a static network. On the other hand, we expect a great advantage from having sequential observations, rather than just single-image observations, in many practical applications. In such cases, the dynamic use of multiple observations will contribute to improving system performance. The face-pose estimation method proposed in this paper, therefore, focuses on an incremental personalization mechanism. The method consists of two parts: a pose-estimation network and an incremental estimation of the face-shape parameters (shape-estimation network). Face poses are estimated from input images and face-shape parameters through the pose-estimation network. The shape parameters are estimated as the output of the shape-estimation network and iteratively updated in a sequence of image observations. Experimental results suggest the effectiveness of using face-shape parameters in face-posture estimation. We also describe the incremental refinement of face-shape parameters using a shape-estimation network.