Measuring service-specific performance and educational value within a general surgery residency: The power of a prospective, anonymous, Web-based rotation evaluation system in the optimization of resident satisfaction

Abstract Background. We used a Web-based evaluation system to institute specific changes to various clinical teaching services in our integrated residency in an effort to optimize the overall quality of the educational experience and measured the resident satisfaction in these rotations. Methods. Residents rated 8 categories of experience on a scale of 1 to 5 (maximum summation score, 40 points). Data were analyzed by t -test for equality of means. A probability value of less than.05 was considered significant. Results. Compliance with completion of the evaluations was 100%. The Chronbach's alpha reliability coefficient of the tool was 0.826. Tukey's estimate of power to achieve additivity was 1.5. Six under-performing services were re-engineered with prominent effects on 7 postgraduate year (PGY) rotations. On 2 general surgery services at 1 hospital, the workload was redistributed, and a dedicated team teaching time was instituted (PGY-3 [a]: before, 22 points/after, 31 points; P =.003; PGY-3 [b]: before, 25 points/after, 31 points; P =.004; PGY-1: before, 24 points/after, 29 points; P =.07). A general surgery service at another hospital redistributed coverage of the attending surgeons to create a nonteaching service (PGY-1: before, 22 points/-after, 27 points; P =.01). The transplantation service (PGY-3) was examined, and the role of the point was redefined (before, 24 points/after, 31 points; P =.01). One vascular service (PGY-2) redistributed cases and workload (before, 27 points/after, 22 points; P =.07). The vascular PGY-2 position was eliminated and replaced by a mid-level practitioner. The cardiothoracic service (PGY-1) rotation was converted into a preceptorship (before, 23 points/after, 30 points; P =.015). Conclusions. A web-based clinical rotation evaluation provides a means for the assessment of the impact of programmatic changes while preserving resident anonymity and maintaining accountability. (Surgery 2001;130:289-95.)