Teaching Facial Expression Production in Autism: The Serious Game JEMImE

Background: Children with autism spectrum disorder (ASD) show impairment in producing facial expressions adapted to social contexts. Several serious games have been computed to help them dealing with facial expression recognition but very few focused on facial expression production adapted to a given social context. Method: JEMImE is a new serious game which aims to help the player to learn how to produce happiness, anger and sadness in a 3D virtual environment with social situations that should be resolved by producing the correct facial expression. He is guided in the game thanks to facial expression feedback and gauges that help him evaluating the quality of his/her production in real time. The feedbacks on the children productions are timely given by a facial expression recognition algorithm integrated in JEMImE architecture. Specific attention was paid to the visual and motivational aspects of the game. Using a brief feasibility study with children with ASD (N = 23), we evaluated the impression of the players on the game aspect and the possibility to insert algorithmic feedbacks in real time inside JEMImE. Results: During the training phase, children with ASD showed a significant progression during training for facial expression production after algorithmic autonomous feedbacks. This means children understand the challenge and that the algorithmic feedbacks are transparent enough to allow gaming. They expressed an overall good subjective experience with JEMImE in terms of ergonomics, playability, visual aspect and motivation. Conclusion: We conclude that the beta-version of JEMImE shows promising potential and that research should proceed on computing more games and scenarios to offer a longer game exposure to children to allow adequate clinical validation.