The Relative Performance Index: Neutralizing Simpson's Paradox

AbstractComparing populations on one or more variables is often of interest. These comparisons are typically made using the mean; however, it is well known that mean comparisons can lead to misinterpretation because of Simpson's Paradox. Simpson's Paradox occurs when there is a differential distribution of subpopulations across the populations being compared and the means of those subpopulations are different. The current paper develops the Relative Performance Index (RPI) to ameliorate effects of Simpson's Paradox. Data from the National Assessment of Educational Progress (NAEP) are used to illustrate use of the new index. The utility of RPI is compared to the population mean and a prior index, the Balanced Index. The paper shows how RPI can be generalized to a variety of contexts with implications for decision making.