External validation of an artificial intelligence solution for the detection of elbow fractures and joint effusions in children

Purpose: The purpose of this study was to conduct an external validation of an artificial intelligence (AI) solution for the detection of elbow fractures and joint effusions using radiographs from a real-life cohort of children.

Materials and methods: This single-center retrospective study was conducted on 758 radiographic sets (1637 images) obtained from consecutive emergency room visits of 712 children (mean age, 7.27 ± 3.97 [standard deviation] years; age range, 7 months and 10 days to 15 years and 10 months), referred for a trauma of the elbow. For each set, fracture and/or effusion detection by eleven senior radiologists (reference standard) and AI solution was recorded. Diagnostic performance of the AI solution was measured via four different approaches: fracture detection (presence/absence of fracture as binary variable), fracture enumeration, fracture localization and lesion detection (fracture and/or a joint effusion used as constructed binary variable).

Results: The sensitivity of the AI solution for each of the four approaches was >89%. Greatest sensitivity of the AI solution was obtained for lesion detection (95.0%; 95% confidence interval: 92.1-96.9). The specificity of the AI solution ranged between 63% (for lesion detection) and 77% (for fracture detection). For all four approaches, the negative predictive values were >92% and the positive predictive values ranged between 54% (for fracture enumeration and localization) and 73% (for lesion detection). Specificity was lower for plastered children for all approaches (P < 0.001).

Conclusion: The AI solution demonstrates high performances for detecting elbow's fracture and/or joint effusion in children. However, in our context of use, 8% of the radiographic sets ruled-out by the algorithm concerned children with a genuine traumatic elbow lesion.