Benchmarking the Quality-monitoring Process: A Comparison of Outcomes Analysis by Trauma and Injury Severity Score (TRISS) Methodology with the Peer-review Process

Background: One measure of optimal function within a trauma center is the ability to critically examine outcomes from the process of care within the institution, yet guidelines for evaluation of the peer-review process are lacking. This study was conducted to determine the correlation between mortality analysis performed by the peer-review process (PR) within a trauma division and outcome analysis as determined by Trauma and Injury Severity Score (TRISS) methodology. Methods: The mortality peer-review data for an entire year at our level I trauma center served as the study population. Information was obtained on probability of survival, and a determination of preventability was made using standard, preexisting criteria. Peer review involves assigning each outcome to a specific category through the process of multidisciplinary assessment. Probability of survival data was not used for this purpose. Kappa analysis was performed to determine the degree of agreement in each category and then tested for significance. Results: One hundred four deaths in 1,868 trauma patients (5.5%) were reviewed at our multidisciplinary conference. Outcomes were judged as preventable, potentially preventable, or nonpreventable. Death directly related to exsanguination was typically categorized as potentially preventable. Kappa analysis demonstrated the greatest agreement between PR and TRISS in the nonpreventable category (kappa = 0.213) and the least agreement in the potentially preventable category (kappa = -0.197). Overall, the kappa Z statistic was nonsignificant (Z = 1.24). Conclusions: Multidisciplinary peer-review outcomes analysis is at least as effective as the computer-generated TRISS probability of survival data for evaluating quality of care in a trauma center and may be more effective for analysis of potentially preventable outcomes.