Development of machine learning model for predicting hospitalization in the prehospital setting

Background: Studies have developed models for predicting patient outcomes for successful risk stratification in the prehospital setting. However, these models generally require many predictors to achieve high prediction ability, resulting in a bar for implementing models in the real clinical setting. Objective: We aimed to develop a simple and implementable machine learning model using automatically-collected data (age, sex, vital signs) to predict patient outcomes during transportation in comparison with National Early Warning Score (NEWS). Methods: This is a retrospective cohort study using data from the ED of three tertiary care hospitals in Japan from April 2017 to March 2020. We included adult patients (aged over 18 years) who were transported to the ED of participating hospitals. We excluded patients with trauma/injury, cardiac arrest, transferred from other hospitals, patients with missing vital signs data, or having data of obvious outliers. The predictors were patient age, sex, mental status evaluated with Japan Coma Scale, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, and oxygen saturation. The primary outcome was hospitalization. We developed a model using XGBoost. Results: During the study period, 3528 visits transported by emergency medical services were eligible. The median NEWS was 4.0, and 2081 patients were hospitalized. The discrimination ability of the newly developed model was 0.70 (95%CI 0.67-0.73), which was better than those of NEWS 0.64 (95%CI 0.61-0.68). The newly developed models performance measures (e.g., sensitivity, specificity) were comparable with NEWS. Conclusions: Our newly developed machine learning model using routinely available data has moderate prediction ability and was better than NEWS.