LSI-Based taxonomy generation: the taxonomist system

The following presents a method for constructing taxonomies by utilizing the Latent Semantic Indexing (LSI) technique. The LSI technique enables representation of textual data in a vector space, facilitates access to all documents and terms by contextual queries, and allows for text comparisons. A taxonomy generator downloads collection of documents, creates document clusters, assigns titles to clusters, and organizes the clusters in a hierarchy. The nodes in the hierarchy are ordered from general to specific in the depth of the hierarchy, and from most similar to least similar in the breadth of the hierarchy. This method is capable of producing meaningful classifications in a short time.