What the eyes reveal: Investigating the detection of automation failures

Abstract In order to detect automation failures in a timely manner, operators are required to monitor automated systems efficiently. The present study analysed eye movements to predict whether or not participants could detect an automation failure. Eye movements were recorded whilst 101 participants were monitoring an automated system where automation failures occurred at irregular intervals. A main result is that about 75.6 per cent of automation failure detections were predicted successfully by the corresponding eye movements. In cases where failures were detected successfully, relevant information is monitored more often and more intensively, in particular shortly before an automation failure happens and while it is happening. The findings are discussed in the context of the personnel selection and training of aviation operatives, as well as incident reporting as used in air traffic control (ATC).