A Simulator-Based Tool That Assesses Pediatric Resident Resuscitation Competency

BACKGROUND.Competency in pediatric resuscitation is an essential goal of pediatric residency training. Both the exigencies of patient care and the Accreditation Council for Graduate Medical Education require assessment of this competency. Although there are standard courses in pediatric resuscitation, no published, validated assessment tool exists for pediatric resuscitation competency. OBJECTIVE. The purpose of this work was to develop a simulation-based tool for the assessment of pediatric residents’ resuscitation competency and to evaluate the tool’s reliability and preliminarily its validity in a pilot study. METHODS.We developed a 72-question yes-or-no questionnaire, the Tool for Resuscitation Assessment Using Computerized Simulation, representing 4 domains of resuscitation competency: basic resuscitation, airway support, circulation and arrhythmia management, and leadership behavior. We enrolled 25 subjects at each of 5 different training levels who all participated in 3 standardized code scenarios using the Laerdal SimMan universal patient simulator. Performances were videotaped and then reviewed by 2 independent expert raters. RESULTS. The final version of the tool is presented. The intraclass correlation coefficient between the 2 raters ranged from 0.70 to 0.76 for the 4 domain scores and was 0.80 for the overall summary score. Between the 2 raters, the mean percent exact agreement across items in each domain ranged from 81.0% to 85.1% and averaged 82.1% across all of the items in the tool. Across subject groups, there was a trend toward increasing scores with increased training, which was statistically significant for the airway and summary scores. CONCLUSIONS. In this pilot study, the Tool for Resuscitation Assessment Using Computerized Simulation demonstrated good interrater reliability within each domain and for summary scores. Performance analysis shows trends toward improvement with increasing years of training, providing preliminary construct validity. C IN PEDIATRIC resuscitation is an essential objective of pediatric residency training.1 It is mandated not only by residency review committee regulations2 but by the exigencies of patient care, given low survival rates from pediatric cardiopulmonary arrest.1,3,4 In September 1999, the Accreditation Council for Graduate Medical Education (ACGME) formally identified and endorsed 6 general competencies for medical education, among them competencies in direct patient care, medical knowledge, and professionalism; residency review committees mandated the implementation and assessment of these competencies as training program requirements effective July 2002. The current evolution of physician assessment is toward performance evaluation, symbolized by the pyramid described by Miller,5 emphasizing a progression from “knows” to “knows how,” “shows how,” and “does.” This highest level corresponds with competency in the performance of a skill, beyond and distinct from the knowledge of how to do so. Because resuscitation is a rare event in pediatrics, residency curricula must rely on simulated experiences rather www.pediatrics.org/cgi/doi/10.1542/ peds.2005-1259 doi:10.1542/peds.2005-1259 Results from this study were presented at the annual meeting of the Pediatric Academic Societies; May 3, 2004; San Francisco, CA.