Pronunciation Training through Articulation Motion Sensing

Vowels are considered the essence of the syllable, which controls the articulation of each word uttered. However, articulation sensing has not been adequately evaluated. The challenging task is that the speech signal contains insufficient information for articulation analysis. It is difficult for users to improve their pronunciation only by getting scoring feedback on pronunciation assessments. We propose a new method to simultaneously use two different acoustic signals (speech and ultrasonic) to recognize lip shape and tongue position. The system gives articulation feedback to a user, identifying the articulation of monophthongs in multiple languages. The proposed technique is implemented into an off-the-shelf smartphone to be more accessible.