Preliminary Study of Mobile Device-Based Speech Enhancement System Using Lip-Reading

Inconspicuous speech enhancement system for laryngectomies using lip-reading is proposed to improve the usability and the speech quality. The proposed system uses a tiny camera on mobile phone and recognize the vowel sequences using lip-reading function. Three types of Japanese vowel recognition algorithms using MLP, CNN, and MobileNets, were investigated. 3,000 image datasets for training and testing were prepared from five persons while uttering discrete vowels. Our preliminary experimental result shows that the MobileNets is appropriate for embedding mobile devices in consideration of a performance both recognition accuracy and calculation cost.