Automated visual identification of characters in situation comedies

The objectives of the work described in this paper are simply stated: given examples of a particular person and an unlabelled video, we wish to find every instance of that person in the video and in others. This is an extremely difficult problem because of the many sources of variation in the person's appearance. We present a two stage approach. A 3-D ellipsoid approximation of the person's head is used to train a set of generative parts-based 'constellation' models which propose candidate detections in an image. The detected parts are then used to align the model, and the detections verified by global appearance. Novel aspects of the approach include the minimal supervision required and the generalization across a wide range of pose. We demonstrate results of detecting three characters in a TV situation comedy.