Improvement of Chinese Word Segmentation Based on Combination Method

How to deal with ambiguity in the segmentation process is a challenging issue that requires Chinese word segmentation algorithms to solve it. This paper proposes an improved dictionary and statisticsbased Chinese word segmentation combination algorithm that can discovery and solve the crossing ambiguity. This algorithm adopts dual stack structure rather than traditional bidirectional matching method to discover ambiguity with less matching time. Furthermore,the algorithm takes methods "choosing longer word"and "choosing word with maximum probability"respectively to deal with general crossing ambiguity and special crossing ambiguity with equal length. Finally,it was verified by case studies that the proposed algorithm has better accuracy than traditional word segmentation algorithms.