Role-based identity recognition for telecasts

Semantic queries involving image understanding aspects require the exploitation of multiple clues, namely the (inter-)relations between objects and events across multiple images, the situational context, and the application context. A prominent example for such queries is the identification of individuals in video sequences. Straightforward face recognition approaches require a model of the persons in question and tend to fail in ill conditioned environments. Therefore, an alternative approach is to involve contextual conditions of observations in order to determine the role a person plays in the current context. Due to the strong relation between roles, persons and their identities, knowing either often allows inferring about the other. This paper presents a system that implements this approach: First, robust face detection localizes the actors in the video. By clustering similar face instances the relative frequency of their appearance within a sequence is determined. In combination with a coarse textual annotation manually created by the broadcast station's archivist the roles and consequently the identities can be assigned and labeled in the video. Starting with unambiguous assignments and cascading appropriately most of the persons can be identified and labeled successfully. The feasibility and performance of the role-based person identification is demonstrated on basis of several programs of a popular German TV show, which consists of various elements like interview scenes, games and musical show acts.