External Validation of Two Clinical Prediction Models for Pediatric Pneumonia

Objective: To externally validate two prediction models for pediatric radiographic pneumonia.

Methods: We prospectively evaluated the performance of two prediction models (Pneumonia Risk Score [PRS] and Catalyzing Ambulatory Research in Pneumonia Etiology and Diagnostic Innovations in Emergency Medicine [CARPE DIEM] models) from a prospective convenience sample of children 90 days - 18 years of age from a pediatric emergency department undergoing chest radiography for suspected pneumonia between January 1, 2022, and December 31st, 2023. We evaluated model performance using the original intercepts and coefficients and evaluated for performance changes when performing recalibration and re-estimation procedures.

Results: We included 202 patients (median age 3 years, IQR 1-6 years), of whom radiographic pneumonia was found in 92 (41.0%). The PRS model had an area under the receiver operator characteristic curve of 0.72 (95% confidence interval [CI] 0.64-0.79), which was higher than the CARPE DIEM (0.59; 95% CI 0.51-0.67) (P < 0.01). Using optimal cutpoints, the PRS model showed higher sensitivity (65.2%, 95% CI 54.6-74.9) and specificity (72.7%, 95% CI 63.4-80.8) compared to the CARPE DIEM model (sensitivity 56.5 [95% CI 45.8-66.8]; specificity 60.9 [95% CI 50.2-69.2]). Recalibration and re-estimation of models improved performance, particularly for the CARPE DIEM model, with gains in sensitivity and specificity, and improved calibration.

Conclusion: The PRS model demonstrated better performance than the CARPE DIEM model in predicting radiographic pneumonia. Among children with a high rate of pneumonia, these models did not reach a level of performance sufficient to be used independently of clinical judgment. These findings highlight the need for further validation and improvement of models to enhance their utility.