A Checklist to Help Faculty Assess ACGME Milestones in a Video-Recorded OSCE

ABSTRACT Background  Faculty members need to assess resident performance using the Accreditation Council for Graduate Medical Education Milestones. Objective  In this randomized study we used an objective structured clinical examination (OSCE) around the disclosure of an adverse event to determine whether use of a checklist improved the quality of milestone assessments by faculty. Methods  In 2013, a total of 20 anesthesiology faculty members from 3 institutions were randomized to 2 groups to assess 5 videos of trainees demonstrating advancing levels of competency on the OSCE. One group used milestones alone, and the other used milestones plus a 13-item checklist with behavioral anchors based on ideal performance. We classified faculty ratings as either correct or incorrect with regard to the competency level demonstrated in each video, and then used logistic regression analysis to assess the effect of checklist use on the odds of correct classification. Results  Thirteen of 20 faculty members rated asses...