Effect of audiovisual perceptual training on the perception and production of consonants by Japanese learners of English

Abstract This study investigates whether L2 learners can be trained to make better use of phonetic information from visual cues in their perception of a novel phonemic contrast. It also evaluates the impact of audiovisual perceptual training on the learners’ pronunciation of a novel contrast. The use of visual cues for speech perception was evaluated for two English phonemic contrasts: the /v/–/b/–/p/ labial/labiodental contrast and /l/–/r/ contrast. In the first study, 39 Japanese learners of English were tested on their perception of the /v/–/b/–/p/ distinction in audio, visual and audiovisual modalities, and then undertook ten sessions of either auditory (‘A training’) or audiovisual (‘AV training’) perceptual training before being tested again. AV training was more effective than A training in improving the perception of the labial/labiodental contrast. In a second study, 62 Japanese learners of English were tested on their perception of the /l/–/r/ contrast in audio, visual and audiovisual modalities, and then undertook ten sessions of perceptual training with either auditory stimuli (‘A training’), natural audiovisual stimuli (‘AV Natural training’) or audiovisual stimuli with a synthetic face synchronized to natural speech (‘AV Synthetic training’). Perception of the /l/–/r/ contrast improved in all groups but learners trained audiovisually did not improve more than those trained auditorily. Auditory perception improved most for ‘A training’ learners and performance in the lipreading alone condition improved most for ‘natural AV training’ learners. The learners’ pronunciation of /l/–/r/ improved significantly following perceptual training, and a greater improvement was obtained for the ‘AV Natural training’ group. This study shows that sensitivity to visual cues for non-native phonemic contrasts can be enhanced via audiovisual perceptual training. AV training is more effective than A training when the visual cues to the phonemic contrast are sufficiently salient. Seeing the facial gestures of the talker also leads to a greater improvement in pronunciation, even for contrasts with relatively low visual salience.