Observer Agreement and Reliabilities of Classroom Observational Measures

There appears to be a growing interest in the study of process variables in naturalistic classroom settings. Numerous observation systems have been developed and used in such investigations (Simon & Boyer, 1974). However, there is a corresponding paucity of published research that documents relationships between teaching-learning processes and educationally relevant pupil outcomes (Dunkin & Biddle, 1974; Heath & Nielson, 1974; Rosenshine, 1971; Smith, 1971; Soar, 1972). A reason for many insignificant findings may be that investigators have inadequately controlled or accounted for a number of sources of variance associated with observational studies (McGaw, Wardrop, & Bunda, 1972). The purpose of this paper is to clarify a number of common methodological errors made in observational studies and then to suggest practical means of minimizing such errors. Of primary concern are those errors introduced by observers and investigators that can limit reliabilities of observational measures. Prevalent confusion concerning the reliabilities of observational measures can be traced to the failure to separate two statistically related