Assessment Linked to Middle School Science Learning Goals: Using Pilot Testing in Item Development

Introduction In this paper we discuss a process for developing distractor-driven, multiple-choice test items (Sadler, 1998) that are closely aligned with science content standards. The work is part of a multi-year, NSF-funded project to develop assessment items aligned to middle school science content standards from AAAS’s Benchmarks for Science Literacy (American Association for the Advancement of Science [AAAS], 1993) and the NRC’s National Science Education Standards (National Research Council [NRC], 1996). We focus in particular on (1) how feedback obtained from students through pilot testing aids in the development of items that are effectively aligned to content standards and (2) how that feedback provides information on students’ thinking about the targeted ideas. The goal of this work is to (1) promote the development of assessment tools and models that can be used by test developers, classroom teachers, and science education researchers to assess student understanding of key ideas in science and identify gaps in students’ knowledge that stand in the way of their understanding phenomena in the world around them, and (2) use assessment to encourage instruction that emphasizes students’ conceptual understanding of the natural world rather than instruction that asks students to memorize facts, definitions, and abstract principles disconnected from the world around them.