Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression

Abstract Smartphone-based devices are increasingly recognized to assess disease symptoms in daily life (e.g. ecological momentary assessment, EMA). Despite this development in digital psychiatry, clinical trials are mainly based on point assessments of psychopathology. This study investigated expectable increases in statistical power by intense assessment in randomized controlled trials (RCTs). A simulation study, based on three scenarios and several empirical data sets, estimated power gains of two- or fivefold pre-post-assessment. For each condition, data sets of various effect sizes were generated, and AN(C)OVAs were applied to the sample of interest (N = 50–N = 200). Power increases ranged from 6% to 92%, with higher gains in more underpowered scenarios and with higher number of repeated assessments. ANCOVA profited from a more precise estimation of the baseline covariate, resulting in additional gains in statistical power. Fivefold pre-post EMA resulted in highest absolute statistical power and clearly outperformed traditional questionnaire assessments. For example, ANCOVA of automatized PHQ-9 questionnaire data resulted in absolute power of 55 (for N = 200 and d = 0.3). Fivefold EMA, however, resulted in power of 88.9. Non-parametric and multi-level analyses resulted in comparable outcomes. Besides providing psychological treatment, digital mental health can help optimizing sensitivity in RCT-based research. Intense assessment appears advisable whenever small sample sizes or small treatment effects are expected, or when applying optimization problems (e.g. machine learning). Simulations for various effects and a short guide for popular power software are provided for study planning. While feasibility of weekly assessment is established, the strategy of pre-post-EMA to boost statistical power needs to be further tested.