Wavelet ANOVA bisection method for identifying simulation model bias

Abstract High-resolution computer models can simulate complex systems and processes in order to evaluate a solution quickly and inexpensively. Many simulation models produce dynamic functional output, such as a set of time-series data generated during a process. These computer models require verification and validation (V&V) to assess the correctness of these simulations. In particular, the model validation effort evaluates if the model is an appropriate representation of the real-world system that it is meant to simulate. However, when assessing a model capable of generating functional output, it is useful to learn more than simply whether the model is valid or invalid. Specifically, if the model is deemed invalid, then what aspects of the model are incorrect? Is it possible to identify over what range the model data are a poor representation of the system data? Current V&V methods cannot identify these ranges. This paper proposes a wavelet analysis of variance (WANOVA) bisection method that first assesses model validity and can also identify the interval(s) over which the model is biased. The technique is illustrated using several simulation studies. Ultimately, this new method supports and expands the efficacy of model validation efforts.