How reliable is eye movement data

Eye data quality varies greatly with the eyetracking system, the individual characteristics of participants, the recording environment, and the operator. The quality of the signal has important repercussions for which eye movement measures are valid, and what conclusions can be drawn from them. Reading research often has high demands in terms of spatial precision and accuracy. Reliably detecting events (e.g., fixations, saccades, pursuit, blinks) depends strongly on intrinsic instrument noise and the correct application of parsing algorithms. We define standard measures of data quality across commercial systems and screen area, as well as system robustness to individual variation, from a large data collection. We show differences in the results from parsing algorithms with varying eye data quality, and infer predictive models of data quality as a function of employed systems, operator, and characteristics of the recorded eye across 12 tower mounted and remote eyetracking systems, and 194 participants. We discuss these results in terms of stimulus constraints in reading research. This work is output of the Eye Data Quality Standardization Committee – a collaboration of manufacturers and researchers towards unbiased measures, and the provision of open-source tools for transparency of eye data quality. (Less)