Evaluation of Computer-Aided Nodule Assessment and Risk Yield (CANARY) in Korean patients for prediction of invasiveness of ground-glass opacity nodule

PLoS One. 2021 Jun 14;16(6):e0253204. doi: 10.1371/journal.pone.0253204. eCollection 2021.

Abstract

Differentiating the invasiveness of ground-glass nodules (GGN) is clinically important, and several institutions have attempted to develop their own solutions by using computed tomography images. The purpose of this study is to evaluate Computer-Aided Analysis of Risk Yield (CANARY), a validated virtual biopsy and risk-stratification machine-learning tool for lung adenocarcinomas, in a Korean patient population. To this end, a total of 380 GGNs from 360 patients who underwent pulmonary resection in a single institution were reviewed. Based on the Score Indicative of Lung Cancer Aggression (SILA), a quantitative indicator of CANARY analysis results, all of the GGNs were classified as "indolent" (atypical adenomatous hyperplasia, adenocarcinomas in situ, or minimally invasive adenocarcinoma) or "invasive" (invasive adenocarcinoma) and compared with the pathology reports. By considering the possibility of uneven class distribution, statistical analysis was performed on the 1) entire cohort and 2) randomly extracted six sets of class-balanced samples. For each trial, the optimal cutoff SILA was obtained from the receiver operating characteristic curve. The classification results were evaluated using several binary classification metrics. Of a total of 380 GGNs, the mean SILA for 65 (17.1%) indolent and 315 (82.9%) invasive lesions were 0.195±0.124 and 0.391±0.208 (p < 0.0001). The area under the curve (AUC) of each trial was 0.814 and 0.809, with an optimal threshold SILA of 0.229 for both. The macro F1-score and geometric mean were found to be 0.675 and 0.745 for the entire cohort, while both scored 0.741 in the class-equalized dataset. From these results, CANARY could be confirmed acceptable in classifying GGN for Korean patients after the cutoff SILA was calibrated. We found that adjusting the cutoff SILA is needed to use CANARY in other countries or races, and geometric mean could be more objective than F1-score or AUC in the binary classification of imbalanced data.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adenocarcinoma of Lung / diagnosis*
  • Adenocarcinoma of Lung / diagnostic imaging
  • Adenocarcinoma of Lung / epidemiology
  • Adenocarcinoma of Lung / pathology
  • Aged
  • Biopsy
  • Diagnosis, Computer-Assisted / methods
  • Female
  • Humans
  • Hyperplasia / diagnosis*
  • Hyperplasia / diagnostic imaging
  • Hyperplasia / epidemiology
  • Hyperplasia / pathology
  • Machine Learning
  • Male
  • Middle Aged
  • Neoplasm Invasiveness
  • Precancerous Conditions / diagnosis*
  • Precancerous Conditions / diagnostic imaging
  • Precancerous Conditions / epidemiology
  • Precancerous Conditions / pathology
  • Republic of Korea / epidemiology
  • Risk Assessment
  • Tomography, X-Ray Computed

Grants and funding

This work was supported by the Korea Medical Device Development Fund grant funded by the Korea government (https://www.kmdf.org/, the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, Republic of Korea, the Ministry of Food and Drug Safety) (Project Number: 202012E01-03) and the faculty research grant of Yonsei University College of Medicine (https://medicine.yonsei.ac.kr/medicine/index.do, 6-2019-0095). The funders had no role in study design, data collection, and analysis, decision to publish, or preparation of the manuscript.