Chapter 13 - Joint Audio-Visual Processing for Video Copy Detection

Abstract Video copy detection is essential for a spectrum of applications, including video search, monitoring, as well as copyright infringement tracking. With enormous growth in the volume of content available and the need to find it quickly, new applications demand robust and efficient underlying video copy detection algorithms. This chapter reviews the recent progress in this area, specifically, the audio and visual alone methods as well as the joint audio and visual approaches. More detailed coverage is given to a video copy detection system built by the authors at AT&T Labs. The system is composed of audio- and visual-based video copy detection submodules, where a hash-based indexing and search engine is employed for efficient content search. A late audio and visual fusion scheme is adopted for combining the copy detection results from both modalities to achieve more robust and accurate results. This system was evaluated in recent TRECVID large-scale copy detection tasks as well as consumer applications aiding personal library management and product search.