Big Changes in Achievement Between Cohorts: A True Reflection of Educational Improvement or Is the Test to Blame?

Large-scale assessments aim to monitor changes in educational sectors by testing students at set intervals. When significant changes in achievement occur between cycles of participation, questions arise as to whether the changes indicate true improvement or can be attributed to aspects of the study, such as the instrument properties. The assessment instruments rely on the assumption of measurement invariance. This chapter demonstrates the application of Rasch theory to investigate measurement invariance and the degree thereof. The participation of South Africa in The Progress in International Reading Literacy Study (PIRLS) is used as an exemplar. Between the 2006 and 2016 cycles of testing, an upward shift of one standard deviation was found for reading literacy achievement of students who wrote the test in isiZulu in the fifth grade. Evidence from Rasch applications for assessing measurement invariance in the cross-national achievement survey with regard to South African participants is examined and the implications for future assessments and educational monitoring are discussed. The contribution of Rasch theory was to provide evidence of internal measurement invariance in large-scale assessments between cohorts and the degree of invariance achieved. The article concludes that Rasch models offer sufficient evidence of internal measurement invariance.