The comparison of auditory, tactile, and multimodal warnings for the effective communication of unexpected events during an automated driving scenario

Abstract In an automated car, users can fully engage in a distractor task, making it a primary task. Compared to manual driving, drivers can engage in tasks that are difficult to interrupt and of higher demand, the consequences can be a reduced perception of, and an impaired reaction to, warnings. In this study we compared three in-vehicle warnings (auditory, tactile, and auditory-tactile) which were presented during three highly attention capturing tasks (visual, auditory, and tactile) while the user was engaged in a self-driving car scenario, culminating in an emergency brake event where the warning was presented. The novel addition for this paper was that three set paced, attention capturing tasks, as well the three warnings were all designed in a pilot study to have comparable workload and noticeability. This enabled a direct comparison of human performance to be made between each of the attention capturing tasks, which are designed to occupy only one specific modality (auditory, visual or haptic), but remain similar in overall task demand. Results from the study showed reaction times to the tactile warning (for the emergency braking event) were significantly slower compared to the auditory and auditory-tactile (aka multimodal or multisensory) warning. Despite the similar reaction times between the in-vehicle auditory warning and the multimodal warning, the multimodal warning led to a reduced number of missed warnings and fewer false responses. However, the auditory and auditory-tactile warnings were rated significantly more startling than the tactile alone. Our results extend the literature regarding the performance benefits of multimodal warnings by comparing them with in-vehicle auditory warnings in an autonomous driving context. The set-pace attention capturing tasks in this study would be of interest to other researchers to evaluate the interaction in an automated driving context, particularly with hard to interrupt and attention capturing tasks.