Concordance Study Between IBM Watson for Oncology and Clinical Practice for Patients with Cancer in China

Background: IBM Watson for Oncology (WFO), which can use natural language processing to evaluate data in structured and unstructured formats, has begun to be used in China. It provides physicians with evidence-based treatment options and ranks them in three categories for treatment decision support. This study was designed to examine the concordance between the treatment recommendation proposed by WFO and actual clinical decisions by oncologists in our cancer center, which would reflect the differences of cancer treatment between China and the U.S.

Patients and methods: Retrospective data from 362 patients with cancer were ingested into WFO from April 2017 to October 2017. WFO recommendations were provided in three categories: recommended, for consideration, and not recommended. Concordance was analyzed by comparing the treatment decisions proposed by WFO with those of the multidisciplinary tumor board. Concordance was achieved when the oncologists' treatment decisions were in the recommended or for consideration categories in WFO.

Results: Ovarian cancer showed the highest concordance, which was 96%. Lung cancer and breast cancer obtained a concordance of slightly above 80%. The concordance of rectal cancer was 74%, whereas colon cancer and cervical cancer showed the same concordance of 64%. In particular, the concordance of gastric cancer was very low, only 12%, and 88% of cases were under physicians choice.

Conclusion: Different cancer types showed different concordances, and only gastric cancers were significantly less likely to be concordant. Incidence and pharmaceuticals may be the major cause of discordance. To be comprehensively and rapidly applied in China, WFO needs to accelerate localization. ClinicalTrials.gov Identifier: NCT03400514.

Implications for practice: IBM Watson for Oncology (WFO) has begun to be used in China. In this study, concordance was examined between the treatment recommendation proposed by WFO and clinical decisions for 362 patients in our cancer center, which could reflect the differences of cancer treatment between China and the U.S. Different cancer types showed different concordances, and only gastric cancers were significantly less likely to be concordant. Incidence and pharmaceuticals may be the major causes of discordance. To be comprehensively and rapidly applied in China, WFO needs to accelerate localization. This study may have a significant effect on application of artificial intelligence systems in China.