Methodological issues in the validation of complex human-machine systems

Integrated system validation is one aspect of the US Nuclear Regulatory Commission`s design review process for human-system interfaces. This paper will consider three methodological issues that must be addressed in validation and their implications for drawing conclusions about the acceptability of the integrated system. They are: representing the integrated system, representing the operational events it must handle, and representing system performance. A logical basis for generalizability from validation tests to predicted performance of the integrated system emerges from the comparability of the psychological and physical processes of the test and actual situations. Generalizability of results is supported when the integrated system, operating conditions and performance are representative of their real-world counterparts. The methodological considerations for establishing representativeness are discussed.