Facial image clustering in stereoscopic videos using double spectral analysis

In this work, we are focusing on facial image clustering techniques applied on stereoscopic videos. We introduce a novel spectral clustering algorithm which combines two well-known algorithms: normalized cuts and spectral clustering. Furthermore, we introduce two approach for evaluating the similarities between facial images, one based on Mutual Information and other based on Local Binary Patterns, combined with facial fiducial points and an image registration procedure. Ways of exploring the extra information available in stereoscopic videos are also introduced. The proposed approaches are successfully tested on three stereoscopic feature films and compared against the state-of-the-art. Author-HighlightsWe developed a facial image clustering algorithm for stereoscopic videos.A double spectral analysis was used for performing the clustering.Features that were used included both global (Mutual Information based) and local (Local Binary Patterns).Facial image trajectory information was also used in clustering.Best results occurred for local features and multiple representative images per facial image trajectory.