Natural Language Processing Accurately Measures Adherence to Best Practice Guidelines for Palliative Care in Trauma

Abstract Context The Trauma Quality Improvement Program Best Practice Guidelines recommend palliative care (PC) concurrent with restorative treatment for patients with life-threatening injuries. Measuring PC delivery is challenging: administrative data is non-specific and manual review is time-intensive. Objective To identify PC delivery to patients with life-threatening trauma and compare the performance of natural language processing (NLP), a form of computer-assisted data abstraction, to administrative coding and gold-standard manual review. Methods Patients ≥18 years admitted with life-threatening trauma were identified from Two Level-I trauma centers (7/2016-6/2017). Four PC process measures were examined during the trauma admission: code-status clarification, goals-of-care (GOC) discussion, PC consult, and hospice assessment. The performance of NLP and administrative coding were compared to manual review. Multivariable regression was used to determine patient and admission factors associated with PC delivery. Results There were 76,791 notes associated with 2,093 admissions. NLP identified PC delivery in 33% of admissions compared to 8% using administrative coding. Using NLP, code-status clarification was most commonly documented (27%), followed by GOC (18%), PC consult (4%), and hospice assessment (4%). Compared to manual review, NLP performed >50 times faster and had a sensitivity of 93%, specificity of 96%, and accuracy of 95%. Administrative coding had a sensitivity of 21%, specificity of 92%, and accuracy of 68%. Factors associated with PC delivery included older age, increased comorbidities, and longer intensive care unit stay. Conclusion NLP performs with similar accuracy to manual review but with improved efficiency. NLP has the potential to accurately identify PC delivery and benchmark performance of best practice guidelines.