Speech recognition system based on visual features and neural network for persons with speech-impairments

The movements of a talker's face, nose, mouth and throat are known to convey visual cues and represent several different kinds of information contained in the speech signals that can improve speech recognition rate, especially where there is noise or hearing-impairment. We proposed a new speech recognition method using these visual features and neural network. Genetic algorithm (GA) was first used to replace steepest descent method (SDM) and make a global search of optimal weight in neural network. The improved GA was then used to train the neural network. Six Chinese vowels were taken as the experimental data. Ten handicapped speakers were taken as the subjects. Recognition experiments show that the method is effective and high speed for speech recognition.