Operator Suspicion and Human-Machine Team Performance Under Mission Scenarios of Unmanned Ground Vehicle Operation

Emergent cyber-attack threats against cyber-physical systems can create potentially catastrophic impacts. The operators must intervene at the right moment when suspected attacks occur, without over-reliance on systems to detect the cyber-attacks. However, military operators are normally trained to trust, rather than suspect systems. We applied suspicion theory to explore how operators detect and respond to cyber-attacks against an unmanned ground vehicle (UGV) system in the operational context of a human–machine team (HMT). We investigated the relationships between the operator suspicion and HMT performance by conducting human-in-the-loop experiments on eight mission scenarios with 32 air-force officers. The experiment yielded a significant, negative relationship between operator suspicion and HMT performance (quantified both in terms of the desirability of decision response and the time to respond). Notably, operator suspicion increased with the combined effects of cyber-attacks and a sentinel alert but not with the alert alone. This finding was particularly meaningful for “false-negative” scenarios, in which no sentinel alert was sent despite cyber-attacks having occurred. Although the operators did not receive an alert, the operators grew more suspicious, seeking more information; it took longer for the operators to respond, and their decision responses were highly divergent (17.2% came with less-desirable responses, and 21.9% were considered instances of over-reliance). In contrast, in “false-positive” scenarios, 95.3% of the operator responses were highly desirable. This experiment has implications for the role of a sentinel alert in engineering trustworthy HMT systems so that the operators can quickly transition through state-suspicion to the most desirable decision.