Test Score Gap Robustness to Scaling

Social scientists frequently rely on the cardinal comparability of test scores to assess achievement gaps between population subgroups and their evolution over time. This approach has been criticized due to the ordinal nature of test scores and the sensibility of results to order-preserving transformations, which are theoretically plausible. Bond and Lang (2013) document the sensitivity of measured ability to scaling choices and develop a method to assess the robustness of changes in ability over time to scaling choices. This paper presents the scale transformation command, which expands the Bond and Lang method to more general cases and optimizes their algorithm to work with large data sets. The program assesses the robustness of an achievement gap between two subgroups to any arbitrary choice of scale by finding bounds for the original gap estimation. Additionally, the program finds scale transformations that are very likely and unlikely to benchmark against the results obtained. Finally, the program also allows the user to measure how much gap growth coefficients change when including controls in their specifications.