Adoption of a diagnostic certainty scale in abdominal imaging: 2-year experience at an academic institution

Purpose: Assess use of a diagnostic certainty scale (CS) for abdominal imaging reports and identify factors associated with greater adoption.

Methods: This retrospective, Institutional Review Board-exempt study was conducted at an academic health system. Abdominal radiology reports containing diagnostic certainty phrases (DCPs) generated 4/1/2019-3/31/2021 were identified by a natural language processing tool. Reports containing DCPs were subdivided into those with/without a CS inserted at the end. Primary outcome was monthly CS use rate in reports containing DCPs. Secondary outcomes were assessment of factors associated with CS use, and usage of recommended DCPs over time. Chi-square test was used to compare proportions; univariable and multivariable regression assessed impact of other variables.

Results: DCPs were used in 81,281/124,501 reports (65.3%). One-month post-implementation, 82/2310 (3.6%) of reports with DCPs contained the CS, increasing to 1862/4644 (40.1%) by study completion (p < 0.001). Multivariable analysis demonstrated reports containing recommended DCPs were more likely to have the CS (Odds Ratio [OR] 4.5; p < 0.001). Using CT as a reference, CS use was lower for ultrasound (OR 0.73; p < 0.001) and X-ray (OR 0.38; p < 0.001). There was substantial inter-radiologist variation in CS use (OR 0.01-26.3, multiple p values).

Conclusion: DCPs are very common in abdominal imaging reports and can be further clarified with CS use. Although voluntary CS adoption increased 13-fold over 2 years, > 50% of reports with DCPs lacked the CS at study's end. More stringent interventions, including embedding the scale in report templates, are likely needed to reduce inter-radiologist variation and decrease ambiguity in conveying diagnostic certainty to referring providers and patients.