Uncertainty-Aware Deep Learning Characterization of Knee Radiographs for Large-Scale Registry Creation

Background: We present an automated image ingestion pipeline for a knee radiography registry, integrating a multilabel image-semantic classifier with conformal prediction-based uncertainty quantification and an object detection model for knee hardware.

Methods: Annotators retrospectively classified 26,000 knee images detailing presence, laterality, prostheses, and radiographic views. They further annotated surgical construct locations in 11,841 knee radiographs. An uncertainty-aware multilabel EfficientNet-based classifier was trained to identify the knee laterality, implants, and radiographic view. A classifier trained with embeddings from the EfficientNet model detected out-of-domain images. An object detection model was trained to identify 20 different knee implants. Model performance was assessed against a held-out internal and an external dataset using per-class F1 score, accuracy, sensitivity, and specificity. Conformal prediction was evaluated with marginal coverage and efficiency.

Results: Classification Model with Conformal Prediction: F1 scores for each label output > 0.98. Coverage of each label output was > 0.99 and the average efficiency was 0.97. Domain Detection Model:The F1 score was 0.99, with precision and recall for knee radiographs of 0.99. Object Detection Model:Mean average precision across all classes was 0.945 and ranged from 0.695 to 1.000. Average precision and recall across all classes were 0.950 and 0.886.

Conclusions: We present a multilabel classifier with domain detection and an object detection model to characterize knee radiographs. Conformal prediction enhances transparency in cases when the model is uncertain.