Towards detecting levels of alertness in drivers using multiple modalities

Distracted and drowsy driving are two very common causes of car accidents as they contribute to 2.3% of all the fatalities caused on the US roads. Therefore, in the era of smart driving there is an increased need of technologies able to monitor driver's alertness and provide timely alerts to the driver. In this paper, we conduct as pilot study and we present a preliminary, yet novel multimodal dataset, collected from 10 subjects using three different modalities. Our modalities include a thermal camera, an RGB camera, and four physiological indicators. The dataset consists of two recording sessions for each subject, thus, offering in total 20 multimodal driving sessions. We propose a machine learning framework aiming to investigate the hypothesis that multimodal features have higher potential towards driver alertness detection. Our dataset and analysis focus on exploring the differences between alertness and drowsiness as they intersect with the presence of different distractions. The results highlight the validity of our hypothesis and introduce interesting future directions for research.