How children with autism spectrum disorder behave and explore the 4-dimensional (spatial 3D + time) environment during a joint attention induction task with a robot

Abstract We aimed to compare, during a joint attention (JA) elicitation task, how children with autism spectrum disorder (ASD) and children with typical development (TD) behave and explore their 4 dimensional (meaning spatial 3D + time) when interacting with a human or with a robotic agent. We built a system that employed a Nao robot and a perception system based on a RGB-D sensor (Kinect) to capture social engagement cues. A JA induction experiment was performed in which children with ASD ( N  = 16) and matched TD children ( N  = 16) had a 3-min interaction with the robot or with a therapist. Nao induced JA by gazing; by gazing and pointing; and by gazing, pointing and vocalizing at pictures. Both groups of children performed well with the therapist. However, with Nao, both groups had lower JA scores, and the children with ASD had a significantly lower score than the TD children. We found that (i) multimodal JA induction was more efficient in both groups; (ii) the 3D spatial world gaze exploration showed less accuracy; and (iii) the trunk position in ASD showed less stability in the 4 dimensions compared to TD controls. We conclude that, in ASD, JA skill depends on the interaction partner, and implies a higher motor and cognitive cost.