The Correlation of Sorted Scores Rule

Abstract The worst performance rule (WPR) states that participants' worst performances on multi-trial tasks, e.g. in reaction time, are more predictive of their general intelligence than are better performances. A common interpretation of the WPR is that poor performances reflect momentary lapses in attention and executive control, which occur more frequently in individuals with low IQ. Here, we first present analyses of simulated data, which indicate that WPR-like phenomena may occur as a function of certain statistical relations between the within-individual variance of multiple test scores and other measures of interest. Specifically, we propose that the WPR is a special case of the more general Correlation of Sorted Scores Rule (CSSR). According to the CSSR, a negative correlation between the test score percentile of sorted scores, e.g. in reaction time, and the test score × construct (e.g. g) correlation, will be seen if the within-individual variance on the tests has a negative correlation with the construct. However, if the latter correlation is positive, the CSSR predicts a positive association also between test score percentiles and test score × construct correlations, i.e. a “best performance rule”. Secondly, we test this hypothesis using analyses in a large cohort (N = 5467) with empirical data on reaction time and intelligence. These results replicate the main findings from the simulation study and, importantly, provide further support for the key prediction of the CSSR, i.e. that samples with a positive correlation between reaction time variance and intelligence tend to show a “best performance rule”.