Assessing physical examination skills using direct observation and volunteer patients.

Background Feedback based on direct observation of the physical examination (PE) is associated with enhanced educational outcomes, yet attending physicians do not frequently observe graduate trainees performing the PE. Methods We recruited volunteer patients (VPs), each with an abnormality of the cardiovascular, respiratory, or neurological system. Interns examined each VP, then presented a differential diagnosis and management plan to two clinician educators, who, themselves, had independently examined the VPs. The clinician educators assessed interns along five domains and provided post-examination feedback and teaching. We collected data on intern performance, faculty inter-rater reliability, correlation with a simulation-based measure of clinical skill, and resident and VP perceptions of the assessment. Results A total of 72 PGY-1 interns from a large academic training program participated. Performance on the cardiovascular and respiratory system was superior to performance on the neurologic exam. There was no correlation between results of an online test and directly observed cardiovascular skill. Interns preferred feedback from the direct observation sessions. VPs and faculty also rated the experience highly. Inter-rater reliability was good for the respiratory exam, but poor for the cardiovascular and neurologic exams. Conclusions Direct observation of trainees provides evidence about PE skill that cannot be obtained via simulation. Clinician educators' ability to provide reliable PE assessment may depend on the portion of the PE being assessed. Our experience highlights the need for ongoing training of clinician educators in direct observation, standard setting, and assessment protocols. This assessment can inform summative or formative assessments of physical exam skill in graduate medical education.