Workload-Based Assessment of a User Interface Design

Lockheed Martin Advanced Technology Laboratories (LM ATL) has designed and developed a tool called Sensor-based Mental Assessment in Real Time (SMART), which uses physiological data to help evaluate human-computer interfaces (HCI). SMART non-intrusively collects and displays objective measures of cognitive workload, visual engagement, distraction and drowsiness while participants interact with HCIs or HCI prototypes. This paper describes a concept validation experiment (CVE) conducted to 1) demonstrate the feasibility of using SMART during user interface evaluations and 2) validate the EEG-based cognitive workload values derived from the SMART system by comparing them to three other measures of cognitive workload (NASA TLX, expert ratings, and expected workload values generated with Design Interactive's Multimodal Information Decision Support tool). Results from the CVE indicate that SMART represents a valuable tool that provides human factors engineers with a non-invasive, non-interrupting, objective method of evaluating cognitive workload.