Entropy-Based Incremental Indexing Term Selection Method for N-gram Full Text Search System

N-gram indexing is the most popular algorithm for the full text search system where each index consists of serial N characters. Especially the full text search for Japanese text usually has the 2-gram characters index as base. The additional higher-gram index is expected to improve the performance. This paper presents the entropy-based method for mining additional indexing terms from DB in order to reduce the waste of AND operation for 2gram.