Do you see what they mean? : An eye-tracking study on the attention for gestures produced by people with aphasia

Introduction People with aphasia (PWA) sometimes try to use alternative means of communication to convey their message. Studies have shown that they produce gestures when speaking, and, more importantly, these can convey information absent in their speech (Hogrefe et al., 2013; Mol et al., 2013). As such, gestures have the potential to contribute to the communication of PWA by helping interlocutors to understand their intended message. Studies on conversations by non-brain damaged participants (NBDP) have shown that interlocutors almost exclusively focus on the speaker’s face while speaking (Beattie et al., 2010; Preisig et al., 2015). However, it remains unclear whether healthy interlocutors do the same when conversing with PWA. Therefore, the present paper investigates up to what degree people pay attention to the gestures produced by PWA. Methods Participants Forty-six native speakers of Dutch (13 male) participated in our study. They were asked to watch videos of individuals (NBDP and PWA) explaining two scenarios. Participants were naive to the purpose of this study. Design Stimuli were short videos of PWA (n=18) and NBDP (n=9) explaining two scenarios from the Scenario Test (van der Meulen et al., 2009). People in the videos explained one of two scenarios: a) buying a sweater, b) having witnessed an accident. The videos used were selected on the criteria that the participants used iconic gestures in their narrations. The videos lasted between 4 and 45 seconds. Participants watched all videos, of which the order was semi-randomized. Eye movements were recorded by means of a remote eye-tracker. Results We analyzed the time that participants looked at the gestures of NBDP and PWA. As the duration of the clips varied, we used proportions. An independent t-test showed that participants looked longer at gestures produced by PWA (M = 0.54, SD = 8.06) than at gestures produced by NBDP (M = 8.97, SD = 7.05), t(25) = -3.67, p = .001. Overall participants looked more at the face than at the gestures, but they looked less at the faces of PWA (M = 69.70, SD = 8.99) than at the faces of NBDP (M = 80.11, SD = 8.76), t(25) = 2.83, p < .01. Discussion Our study provided first evidence in the direction that interlocutors pay more attention to gestures of PWA than those of NBDP. This implies that the gestures produced by PWA are noted by the interlocutor and that they can pick up information conveyed by these gestures. Further research is desirable to determine if interlocutors also tend to understand the message better. At the conference we expect to present further analyses looking into the factors that guide people’s attention to gestures, such as: a) the attention for the gesture by the speakers themselves, but also, b) the comprehensibility of the spoken message.