Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics

ABSTRACT Background  Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging. Objective  We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept. Methods  We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics. Results  Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7],...