Lip Modelling and Segmentation

Lip segmentation is the first step of any audio-visual speech reading system. The accuracy of this segmentation has a major influence on the performances of the global system. But this is a very difficult task. First of all, lip shape can undergo strong deformations during a speech sequence. As many other image processing algorithms, the segmentation task is also influenced by the illumination conditions and by the orientation of the object to be segmented. In this chapter, we present an overview about lip modeling and lip segmentation (region-based and contour-based methods). We limit our study to the problem of lip segmentation in frontal faces. Section I gives an overview about the chrominance information that is used for lip segmentation and a comparison between different chrominance cues is proposed. Section II presents region-based approaches and training steps. Section III focuses on contour-based approaches and parametric lip models. Section IV inventories methods for lip segmentation accuracy evaluation. Some specific applications are briefly presented in section V.