Aligning a 3D headless avatar with a user's 2D face images in real-time

Virtual try-on of clothes has received much attention recently due to its great commercial potential [Hauswiesner et al. 2011 and Meng et al. 2010]. In these applications, a user standing in front of a display mounted with RGBD camera can see him/her in virtual clothes without taking off actual clothes and wearing new ones. The major challenges faced in this regard are alignment of the virtual clothes with the user body, and visibility of the actually worn clothes if the virtual clothes do not completely cover them. To resolve these problems, we replace the user's body below the head by a headless 3D avatar wearing the virtual clothes. The avatar's body is customized to match the user's body size, and its skin color is changed to match the user's skin color, and they are blended together. This not only resolves the above-mentioned issues, but also the virtual clothes appear to be properly wrapping the body as if they are actually worn. Furthermore, minor inaccuracies in tracking are not noticed. All these factors enhance the user experience and create illusion of try-on in front of a mirror. However, success of this approach lies in alignment of real user's head with virtual avatar's body. In this poster, we address this problem.