A systematic approach to developing near real-time performance predictions based on physiological measures

Performance measurements using human sensing and assessment capabilities are limited by an inability to account for the multitude of variables that regulate performance state. Monitoring behavior alone is not adequate for prediction of future performance on a given task and no single physiological measurement can provide a complete assessment that influences performance. Here we investigate how to analyze a range of physiological measurements in near real-time using state of the art signal processing methods to predict performance. We developed multiple predictive computational models to assess when physiology markers that coincide with workload levels are reaching a point that performance decreases or increases may occur in the near future. Traditionally, models will vary significantly between studies (due to the diversity of tasks being tested, the number/type of sensors and differing analysis techniques), leading to specialized models that do not transfer between tasks and individuals. When a model is so specialized that it is only predictive to a specific task and not flexible to inter-or intra-individual differences without complete system retraining, it is impractical in applications outside of controlled experiments. To bring practical use of computational models in real world environments it is important to examine which types of physiological data that can both be reliably processed and analyzed in near real-time and that are highly predictive over time. It is also necessary to minimize the number of sensors in the real world so a sensor and signal sensitivity analysis needs to be performed. We identified and collected physiological signals linked to workload including electroencephalogram (EEG), Heart Rate and Heart Rate Variability (hR/HRV) and Eye-Tracking while performing multiple tasks at varying difficulty levels. We tested a variety of preprocessing methods and computational models, including radial basis function kernel support vector machines and neural networks, to determine predictive power as well as computational time for each type of model. The models were tested using each signal independently as well as combinations of all the signals.