Do students’ judgment models of instructor effectiveness differ by course level, course content, or individual instructor?

Abstract Student evaluations of teaching (SET) are commonly used to evaluate classroom instruction, but their validity in assessing teaching effectiveness is not firmly established. This study investigates a question not previously studied in the literature: the consistency with which students utilize information in evaluating different instructors and courses. Specifically, the study examines students’ relative weightings of six factors in their judgment models for instructor effectiveness. Significant differences ( p ≤0.0001) are found between the models used to evaluate instructors in upper-level and lower-level accounting courses, instructors in cost/managerial and financial accounting courses, and even different instructors teaching the same course. The data are from the SET instrument used at a major state university, and cover a 6-year period. The findings imply that, in comparing and rewarding instructors based on summative evaluations, one needs to be aware of contingencies that affect students’ judgment models. Some of the differences in models are normatively appropriate, but personality or other extraneous environmental variables may inappropriately influence students’ judgment models of instructor effectiveness.