A neurophysiological approach to assess training outcome under stress: A virtual reality experiment of industrial shutdown maintenance using Functional Near-Infrared Spectroscopy (fNIRS)

Abstract Shutdown maintenance, i.e., turning off a facility for a short period for renewal or replacement operations is a highly stressful task. With the limited time and complex operation procedures, human stress is a leading risk. Especially shutdown maintenance workers often need to go through excessive and stressful on-site trainings to digest complex operation information in limited time. The challenge is that workers’ stress status and task performance are hard to predict, as most trainings are only assessed after the shutdown maintenance operation is finished. A proactive assessment or intervention is needed to evaluate workers’ stress status and task performance during the training to enable early warning and interventions. This study proposes a neurophysiological approach to assess workers’ stress status and task performance under different virtual training scenarios. A Virtual Reality (VR) system integrated with the eye-tracking function was developed to simulate the power plant shutdown maintenance operations of replacing a heat exchanger in both normal and stressful scenarios. Meanwhile, a portable neuroimaging device – Functional Near-Infrared Spectroscopy (fNIRS) was also utilized to collect user’s brain activities by measuring hemodynamic responses associated with neuron behavior. A human–subject experiment (n = 16) was conducted to evaluate participants’ neural activity patterns and physiological metrics (gaze movement) related to their stress status and final task performance. Each participant was required to review the operational instructions for a pipe maintenance task for a short period and then perform the task based on their memory in both normal and stressful scenarios. Our experiment results indicated that stressful training had a strong impact on participants’ neural connectivity patterns and final performance, suggesting the use of stressors during training to be an important and useful control factors. We further found significant correlations between gaze movement patterns in review phase and final task performance, and between the neural features and final task performance. In summary, we proposed a variety of supervised machine learning classification models that use the fNIRS data in the review session to estimate individual’s task performance. The classification models were validated with the k-fold (k = 10) cross-validation method. The Random Forest classification model achieved the best average classification accuracy (80.38%) in classifying participants’ task performance compared to other classification models. The contribution of our study is to help establish the knowledge and methodological basis for an early warning and estimating system of the final task performance based on the neurophysiological measures during the training for industrial operations. These findings are expected to provide more evidence about an early performance warning and prediction system based on a hybrid neurophysiological measure method, inspiring the design of a cognition-driven personalized training system for industrial workers.