Biases of Success Rate Differences Shown in Binomial Effect Size Displays.

The intent of a binomial effect size display (BESD) is to show the [real-world] importance of [an] effect indexed by a correlation [r] (R. Rosenthal, 1994, p. 242) by reexpressing this correlation as a success rate difference (SRD) (e.g., treatment group success rate - control group success rate). However, SRDs displayed in BESDs generally overestimate real-world SRDs implied by correlations of (a) dichotomous X and Y variables (Φ coefficients), (b) dichotomous X and continuous Y variables (point-biserial coefficients [r pb s]), and (c) continuous X and Y variables (r xy s). Furthermore, overestimation biases are larger for r xy s than for r pb s. Differences in the sizes of biases linked to different correlations suggest that BESD SRDs reported for different correlations are not comparable. The stochastic difference index (N. Cliff, 1993; A. Vargha & H. D. Delaney, 2000) is recommended as an alternative to the BESD.