Lip Language Recognition System Based on AR Glasses

With the rapid development of science and technology, researches on AR, speech recognition, and lip recognition are more mature. Speech recognition is used in many fields; however, the problem exists in which speech recognition is not accurate in noisy environments. To solve the problem, lip recognition can be used as an aid, but receiving real-time image information is difficult. AR is easy to carry and can obtain image information in real time. But few people currently combine lip recognition with AR. This primary goal of this research is the combination of lip recognition and AR glasses. The image information is obtained through the camera module of the AR glasses, next face detection is performed based on OpenCV, and then, the lip language information is compared with the corpus by image processing to form text and finally reflected on the screen of the AR glasses. We have found that on the premise of reasonable recognition distance, using AR glasses as a carrier, the lip-reading text is displayed in a three-dimensional scene, real-time mapping is realized, and people can see the results of lip recognition more intuitively. It is concluded that lip recognition based on AR glasses works well, is robust, and has strong real-time performance, after a large number of experiments.