An Electronic Medical Record-Based Discharge Disposition Tool Gets Bundle Busted: Decaying Relevance of Clinical Data Accuracy in Machine Learning

Background: Determining discharge disposition after total joint arthroplasty (TJA) has been a challenge. Advances in machine learning (ML) have produced computer models that learn by example to generate predictions on future events. We hypothesized a trained ML algorithm's diagnostic accuracy will be better than that of current predictive tools to predict discharge disposition after primary TJA.

Methods: This study was a retrospective cohort study from a single, tertiary referral center for primary TJA. We trained and validated an artificial neural network (ANN) based on 4368 distinct surgical encounters between 1/1/2013 and 6/28/2016. The ANN's ability to identify discharge disposition was then tested on 1452 distinct surgical encounters between 1/3/17 and 11/30/17.

Results: The area under the curve and accuracy achieved during model validation were 0.973 and 91.7%, respectively, with 25% of patients being discharged to skilled nursing facilities (SNFs). Within our testing data set, 6.7% of patients went to SNFs. The performance in the testing set included an area under the curve of 0.804, accuracy of 61.3%, sensitivity of 28.9%, and specificity of 93.8%.

Conclusions: This is the first prediction tool using an electronic medical record-integrated ANN to predict discharge disposition after TJA based on locally generated data. Dramatically reduced numbers of patients discharged to SNFs due to implementation of a bundled payment model lead to poor recall in the testing model. This model serves as a proof of concept for developing an ML prediction tool using a relatively small data set and subsequent integration into the electronic medical record.