External Validation of the VOCAL‐Penn Cirrhosis Surgical Risk Score in Two Large, Independent Health Systems

BACKGROUND & AIMS Cirrhosis poses an increased risk of post-operative mortality, however it remains challenging to accurately risk stratify patients in clinical practice. The VOCAL-Penn cirrhosis surgical risk score was recently developed and internally validated in the national Veterans Affairs health system, however to date this score has not been evaluated in independent cohorts. The goal of this study was to compare the predictive performance of VOCAL-Penn to the Mayo risk, model for end-stage liver disease (MELD), and MELD-sodium (MELD-Na) scores in two large health systems. APPROACH & RESULTS We performed a retrospective cohort study of patients with cirrhosis undergoing surgical procedures of interest at the Beth Israel Deaconess Medical Center or University of Pennsylvania Health System from 1/1/2008-10/1/2015. The outcomes of interest were 30- and 90-day post-operative mortality. C-statistics, calibration curves, Brier scores, and the index of prediction accuracy (IPA) were compared for each predictive model. RESULTS A total 855 surgical procedures were identified. The VOCAL-Penn score had the numerically highest C-statistic through 90 days post-operative mortality, (e.g., 0.82 vs. 0.79 Mayo vs. 0.78 MELD-Na vs. 0.79 MELD), though differences were not statistically significant. Calibration was excellent for VOCAL-Penn, MELD, and MELD-Na, however the Mayo score consistently overestimated risk. VOCAL-Penn had the lowest Brier score and highest IPA at both timepoints, suggesting superior overall predictive model performance. In subgroup analyses of higher-MELD patients, VOCAL-Penn had significantly higher C-statistics as compared to MELD and MELD-Na. CONCLUSIONS The VOCAL-Penn score (www.vocalpennscore.com) has excellent discrimination and calibration for post-operative mortality among diverse patients with cirrhosis. Overall performance is superior to Mayo, MELD, and MELD-Na scores. In contrast to MELD/MELD-Na, VOCAL-Penn retains excellent discrimination among higher MELD patients.