Quantifying patterns of joint attention during human-robot interactions: An application for autism spectrum disorder assessment

Abstract In this paper we explore the dynamics of Joint Attention (JA) in children with Autism Spectrum Disorder (ASD) during an interaction task with a small humanoid robot. While this robot elicits JA in children, a coupled perception system based on RGB-D sensors is able to capture their behaviours. The proposed system shows the feasibility and the practical benefits of the use of social robots as assessment tools of ASD. We propose a set of measures to describe the behaviour of the children in terms of body and head movements, gazing magnitude, gazing directions (left vs. front vs. right) and kinetic energies. We assessed these metrics by comparing 42 children with ASD and 16 children with typical development (TD) during the JA task with the robot, highlighting significant differences between the two groups. Employing the same metrics, we also assess a subgroup of 14 children with ASD after 6-month of JA training with a serious game. The longitudinal data confirms the relevance of the proposed metrics as they reveal the improvements of children behaviours after several months of training.