ESTABLISHING MINIMUM STANDARDS FOR ESSAYS: BLIND VERSUS INFORMED RÉVIEWS

This study was undertaken to evaluate two procedures for establishing a minimum performance standard for the essay subtest of the National Teacher Examinations (NTE) Communication Skills test. Twenty public school teachers and 20 teacher educators were randomly assigned to either a "blind" or "informed" review panel. Both panels were directed to evaluate the same set of 12 sample essays. Those assigned to the informed panel were apprised of the scores previously awarded to each essay and were asked to decide upon the lowest score they would judge to be acceptable for a beginning teacher. Those assigned to the blind panel were not informed of the scores and were asked to judge whether each essay was at least minimally acceptablefor a beginning teacher. Following their blind reviews, the scores were revealed to this group, and they were also asked to indicate the lowest score they would judge to be acceptable. No significant mean differences were observedfor the standards associated with blind and informed reviews, but a significant mean difference was observed between the standards set by teachers and teacher educators; theformer being more stringent. Based upon criteria other than mean differences, it was concluded that the preferred procedure for setting standards on essays should involve a blind review followed by an informed review. This strategy was subsequently implemented in a state-wide validation study in Virginia, and the results of thisfull-scale study are presented along with results reported for a study using informed reviews in another state. With the nationwide movement toward minimum-competency testing of both students and teachers during the past 10 years, there has been a corresponding increase in the amount written about procedures for establishing minimum performance standards. However, much of what has been written focuses on procedures designed for use with multiple-choice tests, and very little has been written about establishing minimumperformance standards for essays. In some settings, it may be possible to base the