Multivariate Nonparametric Statistical Techniques for Simulation Model Validation

Abstract : This report documents the findings of an Army SBIR Phase 1 study on multivariate nonparametric tests for stochastic model validation. We herein introduce a method for generalizing rank transformations to the multivariate domain such that the rank-transformed set is uniformly distributed in multiple dimensions. This furnishes a more robust hypothesis testing technique than earlier proposed approaches and has certain computational advantages. This approach is well adapted for continuous-output models. For tests based on partitioning the model output space into bins and computing a confidence statistic based directly on bin counts, as opposed to computing statistical moments, we introduce a log-likelihood statistic that appears to be an excellent summary indicator of correspondence between a simulation model and test data. The approach is extremely versatile and well-adapted to discrete-output models.