Biologically relevant simulations for validating risk models under small-sample conditions

In designing scientific experiments, power analysis is too often given a superficial treatment— choice of sample size is often made based on idealized distributions and simplistic tests that do not reflect the real-world constraints under which the actual data will be collected. We have developed a general Monte Carlo framework for two-group comparisons which samples points from a two-dimensional parameter space and at each point generates simulated datasets which are compared to simulated datasets for a “control group” at a fixed point in the parameter space. Rather than uniformly sampling this parameter space, our algorithm rapidly converges on a contour corresponding to the smallest detectable difference for the sample size of interest.