Computerized Mortality Prediction for Community-acquired Pneumonia at 117 VA Medical Centers.

RATIONALE Computerized severity assessment for community-acquired pneumonia (CAP), could improve consistency and reduce clinician burden. OBJECTIVES Develop and compare 30-day mortality prediction models using electronic health record data including a computerized Pneumonia Severity Index (PSI) versus models with additional variables. METHODS Among adults presenting to Emergency Departments (EDs) at 117 VA Medical Centers EDs between 1/1/2006-12/31/2016 with CAP, we compared a computerized score with all variables from the PSI except confusion and pleural effusion ("ePSI score") to 10 novel models employing logistic regression, spline, and machine learning (ml) methods using PSI variables (PSI), age, sex and 26 physiologic variables (PSI-28), and all 69 variables (PSI-69). Models were trained using encounters before 1/1/2015, tested on encounters during and after 1/1/2015, and compared using area under receiver operator curve-confidence intervals and patient event rates at a threshold PSI score of ≤70. RESULTS Among 297,498 encounters, 7% died in 30 days. Compared to ePSI score (AUROC-CI=0.77-0.78), performance increased with model complexity (logPSI=0.79-0.80, mlPSI=0.83-0.85) and number of variables (logPSI-69=0.84-085, mlPSI-69=0.86-0.87). Models limited to age, sex and physiologic variables also demonstrated high performance (mlPSI-28=0.84-0.85). At an ePSI of ≤70 and mortality risk cut-off of <2.7%, the ePSI score identified 31%, mlPSI-28 53%, and mlPSI-69 56% of all patients as "low risk", with similar rates of mortality, hospitalization, and 7-day secondary hospitalization. CONCLUSIONS Computerized versions of the PSI accurately identified pneumonia patients at low risk of death. More complex models classified more patients at low risk of death with similar adverse outcomes.