Multimodal deep learning models utilizing chest X-ray and electronic health record data for predictive screening of acute heart failure in emergency department

Background and objectives: Ambiguity in diagnosing acute heart failure (AHF) leads to inappropriate treatment and potential side effects of rescue medications. To address this issue, this study aimed to use multimodality deep learning models combining chest X-ray (CXR) and electronic health record (EHR) data to screen patients with abnormal N-terminal pro-B-type natriuretic peptide (NT-proBNP) levels in emergency departments.

Methods: Using the open-source dataset MIMIC-IV and MIMICCXR, the study population consisted of 1,432 patients and 1,833 pairs of CXRs and EHRs. We processed the CXRs, extracted relevant features through lung-heart masks, and combined these with the vital signs at triage to predict corresponding NT-proBNP levels.

Results: The proposed method achieved a 0.89 area under the receiver operating characteristic curve by fusing predictions from single-modality models of heart size ratio, radiomic features, CXR, and the region of interest in the CXR. The model can accurately predict dyspneic patients with abnormal NT-proBNP concentrations, allowing physicians to reduce the risks associated with inappropriate treatment.

Conclusion: The study provided new image features related to AHF and offered insights into future research directions. Overall, these models have great potential to improve patient outcomes and reduce risks in emergency departments.