Diagnostics for Stochastic Emulators.

Computer models, also known as simulators, can be computationally expensive to run, and it is is for this reason that statistical surrogates, also known as emulators, have been developed for such computer models. Gaussian processes are commonly used to emulate deterministic simulators, and they have also been extended to emulate stochastic simulators that have input-dependent noise. Any statistical model, including an emulator, should be validated before being used. We discuss how current methods of validating Gaussian process emulators of deterministic models are insufficient when applied to Gaussian process emulators of stochastic computer models. We develop tools for diagnosing problems in such emulators, based on independently validating the mean and variance predictions using out-of-sample, replicated, simulator runs. To illustrate these ideas, we apply them to a simple toy example, and a real example of a building performance simulator.