Comparison of Outlier Detection Methods in NEAT Design

In equating practice, the existence of outliers in the anchor items can deteriorate the equating accuracy and threaten the validity of test scores. This study used simulation to compare the performance of three outlier detection methods when conducting equating: the t-test method, the logit difference method, and the robust z statistic. The investigated factors include sample size, proportion of outliers, item difficulty drift direction, and group difference. Overall, across all simulated conditions, the t-test method outperformed the other methods in terms of sensitivity of flagging true outliers, specificity of flagging true non-outliers, bias of translation constant, and the root mean square error of the estimated examinee ability.