Distractor Efficiency in an Item Pool for a Statistics Classroom Exam: Assessing Its Relation With Item Cognitive Level Classified According to Bloom’s Taxonomy

Multiple-choice items are one of the most commonly used tools for evaluating students’ knowledge and skills. A key aspect in this type of assessment is the presence of functioning distractors, i.e., wrong alternatives intended to be plausibly correct among those students with lower achievement. To our knowledge, no work has investigated the relationship between distractor performance and the complexity of the cognitive task required to give the correct answer. The aim of this study was to investigate this relation, employing the first three levels of Bloom’s Taxonomy. Specifically, it was hypothesized that items classified into a higher level of Bloom’s classification would show a greater number of functioning distractors. The study involved 174 items administered to a sample of 848 undergraduate psychology students during their Statistics exam. Each student received 30 items randomly selected from the 174-item pool. Bivariate results mainly supported the authors’ hypothesis. Effect size was small (2=.024) but relevant. When the relationship was controlled for other item features, it lost statistical significance, partly because of a confounding effect with items’ difficulty.