Deep interaction: Wearable robot-assisted emotion communication for enhancing perception and expression ability of children with Autism Spectrum Disorders

Abstract Recent changes in both the social economy and people’s living conditions and their habits have had an influence on the incidence of Autism Spectrum Disorder (ASD), which has brought huge economic and mental burden to the society , and has become an urgent public health problem. The main symptom of autism in children is the presence of a social barrier, and one of the main reasons for that is a lack of emotional cognitive ability. However, the existing autism-treatment systems intended for children pay little attention to the emotion cognition disorder. Besides, too much importance has been given to the interaction with children, while limiting the timeliness and movability of these systems. With the aim to address this shortcoming, in this work, we focus on the emotion cognition disorder and design a feasible solution for enhancing perception and expression ability of children with ASD. First, the first-view emotional care system for children with ASD (First-ECS) is developed using a wearable robot as a system carrier and realizing the deep emotional interaction with children with autism from the first-view perspective. Emotion communication is used to meet high timeliness requirements for emotion transmission in the First-ECS. Next, the emotional interaction mechanism that is applicable to the line of sight and non-line of sight communication scenarios is introduced. Also, the emotion perception engine and emotion expression engine are designed. Subsequently, multimodal data collection and processing by a wearable affective robot are discussed. In addition, this paper introduces a multimodal data fusion method from the angle of emotion relevance and emotion computing model based on audio-visual data. Finally, a demo platform is built to verify the feasibility of the proposed system.