Measures of interobserver agreement

INTRODUCTION RELIABILITY FOR CONTINUOUS SCALE MEASUREMENTS Model for Reliability Studies Inference Procedures on the Index of Reliability for Case (1) Analysis of Method - Comparison Studies Comparing Reliability Coefficients MEASURES OF 2x2 ASSOCIATION AND AGREEMENT OF CROSS CLASSIFIED DATA Introduction Indices of Adjusted Agreement Cohen's Kappa =Chance Corrected Measure of Agreement Intraclass Kappa The 2x2 Kappa in the Context of Association Stratified Kappa Conceptual issues COEFFICIENTS OF AGREEMENT FOR MULTIPLE RATERS AND MULTIPLE CATEGORIES Introduction Multiple Categories and Two Raters Agreement for Multiple Raters and Dichotomous Classification Multiple Raters and Multiple Categories Testing the Homogeneity of Kappa Statistic from Independent Studies ASSESSING AGREEMENT FROM DEPENDENT Introduction Dependent Dichotomous Assessments Adjusting for Covariates Likelihood Based Approach Estimating Equations Approach Loglinear and Association Models Appendix I: Joint probability distribution of repeated dichotomous assessments Appendix II: Correlation between estimated kappas SAMPLE SIZE REQUIREMENTS FOR THE DESIGN OF RELIABILITY STUDY Introduction The Case of Continuous Measurements The Non-Normal Case Cost Implications The Case of Dichotomous Assessments Bibliography