Language and Speech Architecture for Prototype 1

This invention is a scaleable system to perform exact matching or similarity matching between a large store of reference temporal media sequences and a query target temporal media sequence. The system is not limited to finding exact matching media segments but also can find media segments that are similar. One kind of similarity between media segments is the similarity between a long commercial and a short commercial, where the short commercial is formed by sub-segments of the longer commercial. Another kind of similarity of two media segment is when they depict three-dimensional actions that are similar and imaged from similar viewpoints. Given a reference media segment, a multitude of features are computed in a consistent way from either predetermined or media content-dependent key intervals. These features are stored in segment index tables along with identifiers of the corresponding reference media segments. At search time, again, a multitude of features is computed from each key interval and now the features are used to search the segment index table for identifiers of stored media segments. If the key intervals are similar, many if not all, features computed from the key intervals point to the same reference media identifier.