SEMI-SYNCHRONOUSSPEECHAND PENINPUT

This paper proposes anewinterface method using semi-synchronous speech andpeninput formobile environments. Inthis interface, auserspeaks while writing, wherepeninput complements speech toachieve higher recognition performance thanspeech alone. A multimodal recognition algorithm that canhandle theasynchronisity ofthetwomodesusing asegment-based unification scheme isproposed. Thismethod isevaluated under noisy conditions with four different pen-input interfaces: character, stroke, pen-touch, and point-to-character, eachofwhich isassumed tobegiven foraphrase unit inspeech. Itisconfirmed that therecognition accuracy isimproved bytheproposed method incomparison withthat byspeech alone inall thepen-input conditions. IndexTerms-Userinterfaces, speech recognition, handwrittencharacter recognition