Prediction of high-risk emergency department revisits from a machine-learning algorithm: a proof-of-concept study

Background: High-risk emergency department (ED) revisit is considered an important quality indicator that may reflect an increase in complications and medical burden. However, because of its multidimensional and highly complex nature, this factor has not been comprehensively investigated. This study aimed to predict high-risk ED revisit with a machine-learning (ML) approach.

Methods: This 3-year retrospective cohort study assessed adult patients between January 2019 and December 2021 from National Taiwan University Hospital Hsin-Chu Branch with high-risk ED revisit, defined as hospital or intensive care unit admission after ED return within 72 hours. A total of 150 features were preliminarily screened, and 79 were used in the prediction model. Deep learning, random forest, extreme gradient boosting (XGBoost) and stacked ensemble algorithm were used. The stacked ensemble model combined multiple ML models and performed model stacking as a meta-level algorithm. Confusion matrix, accuracy, sensitivity, specificity and area under the receiver operating characteristic curve (AUROC) were used to evaluate performance.

Results: Analysis was performed for 6282 eligible adult patients: 5025 (80.0%) in the training set and 1257 (20.0%) in the testing set. High-risk ED revisit occurred for 971 (19.3%) of training set patients vs 252 (20.1%) in the testing set. Leading predictors of high-risk ED revisit were age, systolic blood pressure and heart rate. The stacked ensemble model showed more favourable prediction performance (AUROC 0.82) than the other models: deep learning (0.69), random forest (0.78) and XGBoost (0.79). Also, the stacked ensemble model achieved favourable accuracy and specificity.

Conclusion: The stacked ensemble algorithm exhibited better prediction performance in which the predictions were generated from different ML algorithms to optimally maximise the final set of results. Patients with older age and abnormal systolic blood pressure and heart rate at the index ED visit were vulnerable to high-risk ED revisit. Further studies should be conducted to externally validate the model.