The relative effectiveness of concept-based versus content-based video retrieval

Three video search systems were compared in the interactive search task at the TRECVID 2003 workshop: a text-only system, which searched video shots through transcripts; a features-only system, which searched video shots through 16 video content features (e.g., airplanes and people); and a combined system, which searched through both transcripts and content features. 36 participants each completed 12 video search tasks. The hypothesis that the combined system would perform better than both the text-only and the features-only systems was not supported, and large topic effects were found. Further analysis showed that concept-based video retrieval worked best for specific topics, whereas the hybrid retrieval techniques which combine both concept- and content-based video retrieval showed some advantage when searching for generic topics. The results have implications for topic/task analysis for video retrieval research, and also for the implementation of hybrid video retrieval systems.