Multimodal perception of whispered and voiced prosody in French: A preliminary fMRI study

After long being considered to be right-lateralized, the processing of prosody appears to be bilateral. Moreover, prosody is traditionally considered as an auditory process, but we have shown that it is multisensory (Dohen & Loevenbruck, 2009). This fMRI study examines the processing of prosodic contrastive focus (emphasizing a constituent in an utterance). Previous studies (Wildgruber et al. 2004, Tong et al. 2005) have shown the involvement of bilateral or left only temporal and parietal regions and bilateral or right-only frontal regions. In this study, 12 subjects were asked to judge whether part of an utterance was focused or not. The stimuli were presented in 3 modalities: auditorily, audiovisually or visually and in normal (voiced) speech or whispered speech. The event-related pseudo-random design included 4 functional scans (half voiced, half whisper). Each scan included 12 events per condition (6 conditions: 3 modalities x 2 prosodic cases) and 14 null-events. The behavioural results show that subjects were able to identify focus from non-focus cases. The processing of focus vs. baseline in voiced speech yields bilateral auditory activations in BA22, 41-42, and left temporal gyrus, inferior frontal gyrus, supplementary motor area and cingulate gyrus. Whispered speech additionally activates the right BA6. Furthermore, for both voiced and whispered speech, identifying focus vs. non-focus involves the left supramarginal gyrus and the left inferior temporal gyrus (BA37). Perception of prosodic focus (vs. no focus) appears to be essentially processed in left associative areas. This illustrates the necessity of associating various types of information to detect focus.