Upstrap for estimating power and sample size in complex models

SO_SCPLOWUMMARYC_SCPLOWPower and sample size calculation are major components of statistical analyses. The upstrap resampling method introduced by Crainiceanu and Crainiceanu (2018) was proposed as a general solution to this problem but has not been assessed in numerical experiments. We evaluate the power and sample size estimation properties of the upstrap for data sets that are larger or smaller than the original data set. We also expand the scope of upstrap and propose a solution to estimate the power to detect: (1) an effect size observed in the data; and (2) an effect size chosen by a researcher. Simulations include the following scenarios: (a) one- and two-sample t-tests; (b) linear regression with both Gaussian and binary outcomes; (c) multilevel mixed effects models with both Gaussian and binary outcomes. We illustrate the approach using a reanalysis of a cluster randomized trial of malaria transmission. The accompanying software and data are publicly available.