Multimodal information improves the rapid detection of mental fatigue

Abstract One of the major challenge in the detection of mental states is improving the accuracy of brain activity-based detectors with additional information from extracranial signals. We assessed the suitability, for real-time mental fatigue detection, of EEG, EOG and ECG measurements, taken separately or together. Thirteen subjects performed six blocks of switching tasks. For each participant, the block with the lowest error rate from the first two blocks and the block with the highest error rate from the last three blocks were discriminated with a machine learning algorithm (support vector machine). The classification scores obtained with ECG or EOG were greater than would be expected by chance (>50%) for time windows of at least 8 s. EEG was the best single mode of detection, with classification scores ranging from 80 ± 3% with a 4 s time window to 94 ± 2% with a 30 s time window. The addition of ECG and EOG features to EEG features significantly increased classification scores for short time windows (e.g., to 86 ± 3% with a 4 s time window, p