An Automated Mutual Gaze Detection Framework for Social Behavior Assessment in Therapy for Children with Autism

Mutual gaze is one of the most significant, reliable, and observable social cues that we can use for establishing and maintaining successful social interactions. This cue has been actively used to assess the level of social behavior in the context of autism therapy. However, collecting gaze data manually and evaluating them is so challenging, which requires a lot of time and effort from therapy experts. To address these issues, in this paper, we introduce an automated mutual gaze detection framework, grounded based on previous works on automated gaze detection, as an effective predictive model for social visual behavior analysis and assessment in autism therapy. To evaluate the proposed gaze prediction framework, we prepare an in-house video dataset that captures social interactions between children with autism and their therapy trainers (N = 10, 30 video recordings). We estimate the mutual gaze ratio of children using our prediction model, then compared it with the social visual behavior scores that therapy experts manually annotated. The results showed that our framework provided mutual gaze ratio scores that reliably represent (or even replace) the therapy experts’ hand-coded social visual behavior scores through different analysis approaches: descriptive comparisons, correlation analysis, and regression prediction. We report our findings and discuss the implications of the proposed work in the context of visual behavior analysis for children with autism.