Diagnostic Performance of O-RADS US (Version 2019 and Version 2022) Incorporating Acoustic Shadowing by Junior Radiologists: Analyzing 1061 Adnexal Masses

Introduction: Acoustic shadowing is an important benign ultrasound (US) feature for adnexal masses (AMs). To validate the diagnostic performance and interobserver agreement of the 2019 version and 2022 version of Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) and ascertain whether adding acoustic shadowing to O-RADS US v2019 as a benign ultrasound feature can enhance its diagnostic efficacy among junior radiologist.

Methods: This retrospective study included consecutive women with suspected adnexal masses who underwent ultrasound examinations between September 2022 and January 2024. One junior doctor (Reader 1, 2 years of experience) classified each AM according to the O-RADS US v2019 and the v2022. Lesions were reclassified according to the US features of acoustic shadowing based on the O-RADS US v2019: the O-RADS category was downgraded by one level with acoustic shadowing and remained unchanged without acoustic shadowing for categories 2-5. Receiver operating characteristic (ROC) curve analysis was used to evaluate the diagnostic performance of the two versions of O-RADS and the modified O-RADS model. Two independent junior radiologists (Reader 1 and Reader 2 with 3 years of experience) then classified the 200 AMs randomly selected for a test-retest analysis. Kappa (к) statistics were used to assess the interobserver agreement.

Results: Overall, 1015 women (range, 16-86 years) with 1061 AMs were evaluated. Of the lesions, 864 (81.4%) were benign and 197 (18.6%) were malignant. The area under the ROC curve (AUC) for O-RADS v2019 and v2022 were 0.920 (95% confidence interval [CI]: 0.902, 0.936, P < .001) and 0.924 (95% CI: 0.906, 0.939, P < .001), respectively. The modified model based on O-RADS v2019 incorporating acoustic shadowing as a benign US feature showed an improved AUC of 0.934 (95% CI: 0.917, 0.948, P < .001). A significant difference was observed between the AUCs of the modified O-RADS and two versions of O-RADS models (P < .005). The inclusion of acoustic shadowing increased specificity by 5.4% in predicting malignant adnexal masses, compared with the O-RADS US v2019 with a specificity of 76.2%. Using the modified O-RADS category 4 as the optimal cut-off value for predicting malignancy showed a sensitivity, specificity, positive predictive value, and negative predictive value were 94.4% (95% CI: 90.2%, 97.2%), 81.6% (95% CI: 78.8%, 84.1%), 53.9% (95% CI: 50.3%, 57.5%), and 98.5% (95% CI: 97.3%, 99.1%), respectively. The inter-observer agreement in the O-RADS category between these two junior radiologists was good (κ = 0.74, P < .001).

Conclusion: We validated the excellent performance of the Ovarian-Adnexal Reporting and Data system Ultrasound for diagnosing adnexal masses, and the inclusion of acoustic shadowing increased specificity by 5.4% in predicting malignant adnexal masses, compared with the O-RADS US v2019 with the specificity of 76.2%.