Playful Screening of Executive Functions Using Augmented Reality and Gaze Based Assessment

Augmented Reality (AR) technologies have recently been explored for application in dementia care, including cognitive training and screening, navigational assistance to find their way around, assistance to identify friends and family members, and assistance with activities of daily living. The presented work proposes the use of mobile and playful assessment of executive functions in the home environment by AR technology. We implemented a gamified version of the neuropsychological test ‘Tower of London’ (Shallice, 1982 [1]). It is played with poles of various size and colored balls, starting each exercise with a certain start and desired goal configuration of balls on poles. The novel AR-based technology enables hand-based interaction with artificial objects in the field of view and monitoring of the user’s gaze behavior via eye tracking towards the artificial objects of interest. A pilot study was performed with 12 healthy people that played the game with various degrees of difficulty concerning its planning depth. The results of the study demonstrate statistically significant correlation between the eye tracking features and the planning ability as well as cognitive flexibility measured by standardized psychological tests. This study provides first indication that executive functions can be estimated from playful AR-based interaction. Future work will focus on adjusting the game for persons with dementia, such as, by decreasing the difficulty level appropriately.