Investigation of the factors affecting the pre-test effect in national curriculum science assessment development in England

Background: All national curriculum tests in England are pre-tested as part of the development process. Differences in pupil performance between pre-test and live test are consistently found. This difference has been termed the pre-test effect. Understanding the pre-test effect is essential in the test development and selection processes and in the setting of cut scores for the national curriculum tests in England. This study looked at the national science tests for 11-year-olds in England (termed key stage 2). Purpose: The aim of this study was to investigate whether the pre-test effect differs when the item data is divided in three ways: by pupil level, item level and item type. Sample: Second pre-test data was captured from Year 6 (aged 11) pupils (2006: n = 1010 for test A, n = 1002 for test B; 2007: n = 895 for test A, n = 889 for test B) from a representative population of state-funded schools in England as defined by school type, size of year group, region and achievement. Live test data was captur...