Improving English Phoneme Pronunciation with Automatic Speech Recognition Using Voice Chatbot

Computer Aided Pronunciation Training (CAPT) is gaining attention in recent years especially for Second Language (L2) and Foreign Language (FL) learners. Nevertheless, operating such learning tools usually requires some training which may hinder the users’ interest. To address these issues, we propose a Phonics Learning Voice Chatbot (PLVC) using Automatic Speech Recognition (ASR) and Triplet Neural Networks (TNNs). TNNs have excellent capabilities in discerning different entities that are similar unless examined in fine grain scales. This distinctive feature enables them to be utilized in classifying and assessing phonemes. The employment of chatbot in our teaching process not only relieves users from using keyboard, mouse, and menus, the teaching storyline can also be adjusted automatically and imperceptibly based on the user replies during teaching. Finally, PLVC has one unique feature that it can be used to assess the performance of the users in phonic learning. We performed an experiment with 10 users and compared the ratings given by PLVC and experienced phoneme assessors and found that the correlation coefficient is 0.71, showing the assessment done by PLVC is reliable.