Rationale and objectives: We conducted an observer study to investigate whether radiologists can judge similarities in pairs of breast masses and lung nodules consistently and reproducibly.
Materials and methods: Institutional review board approval and informed observer consent were obtained. This study was compliant with the Health Insurance Portability and Accountability Act. We used eight pairs of breast masses on mammograms and eight pairs of lung nodules on computed tomographic images, for which subjective similarity ratings ranging from 0 to 1 were determined in our previous studies. From these, four sets of image pairs were created (ie, a set of eight mass pairs, a set of eight nodule pairs, and two mixed sets of four mass and four nodule pairs). Eight radiologists, including four breast radiologists and four chest radiologists, compared all combinations of the eight pairs in each set using a two-alternative forced-choice (2AFC) method to determine the similarity ranking scores by identifying which pair was more similar than the other pair based on the overall impression for diagnosis.
Results: In the mass set and nodule set, the relationship between the average subjective similarity ratings and the average similarity ranking scores by 2AFC indicated very high correlations (r = 0.91 and 0.88). Moreover, in the two mixed sets, the correlations between the average subjective similarity ratings and the average similarity ranking scores were also very high (r = 0.90 and 0.98). Thus, radiologists were able to compare the similarities for pairs of lesions consistently, even in the unusual comparison of pairs of completely different types of lesions.
Conclusion: The subjective similarity of a pair of lesions in medical images can be quantified consistently by a group of radiologists. The concept of similarity of lesions in medical images can be subjected to rigorous scientific research and investigation in the future.