Power Estimates: The Effect of Dependent Variable Reliability on the Power of One-Factor ANOVAs.

The literature indicates that the power of a statistical test is influenced by the reliability of the dependent variable; however the authors disagree about the direction and strength of this effect. The purpose of this study was to quantify and to tabulate the degree to which reliability affects the power of the one-factor Analysis of Variance (ANOVA) by taking into account sample size, level of significance, and true score effect size. True score effect size is defined as the difference between means in true score standard deviation units without variation due to the inconsistency of an instrument or lack of its reliability. Results confirm that the impact on power is substantial, particularly when sample sizes are less than 100 per group and/or when small or moderate effect size differences are expected. It is suggested that the influence of reliability is of considerable interest to those who include subtest analyses in their work as well as those who study low incidence populations. The tables provid...