Artificial INtelligence to Support Informed DEcision-making (INSIDE) for Improved Literature Analysis in Oncology

Background: Defining optimal therapeutic sequencing strategies in prostate cancer (PC) is challenging and may be assisted by artificial intelligence (AI)-based tools for an analysis of the medical literature.

Objective: To demonstrate that INSIDE PC can help clinicians query the literature on therapeutic sequencing in PC and to develop previously unestablished practices for evaluating the outputs of AI-based support platforms.

Design, setting, and participants: INSIDE PC was developed by customizing PubMed Bidirectional Encoder Representations from Transformers. Publications were ranked and aggregated for relevance using data visualization and analytics. Publications returned by INSIDE PC and PubMed were given normalized discounted cumulative gain (nDCG) scores by PC experts reflecting ranking and relevance.

Intervention: INSIDE PC for AI-based semantic literature analysis.

Outcome measurements and statistical analysis: INSIDE PC was evaluated for relevance and accuracy for three test questions on the efficacy of therapeutic sequencing of systemic therapies in PC.

Results and limitations: In this initial evaluation, INSIDE PC outperformed PubMed for question 1 (novel hormonal therapy [NHT] followed by NHT) for the top five, ten, and 20 publications (nDCG score, +43, +33, and +30 percentage points [pps], respectively). For question 2 (NHT followed by poly [adenosine diphosphate ribose] polymerase inhibitors [PARPi]), INSIDE PC and PubMed performed similarly. For question 3 (NHT or PARPi followed by ¹⁷⁷Lu-prostate-specific membrane antigen-617), INSIDE PC outperformed PubMed for the top five, ten, and 20 publications (+16, +4, and +5 pps, respectively).

Conclusions: We applied INSIDE PC to develop standards for evaluating the performance of AI-based tools for literature extraction. INSIDE PC performed competitively with PubMed and can assist clinicians with therapeutic sequencing in PC.

Patient summary: The medical literature is often very difficult for doctors and patients to search. In this report, we describe INSIDE PC-an artificial intelligence (AI) system created to help search articles published in medical journals and determine the best order of treatments for advanced prostate cancer in a much better time frame. We found that INSIDE PC works as well as another search tool, PubMed, a widely used resource for searching and retrieving articles published in medical journals. Our work with INSIDE PC shows new ways in which AI can be used to search published articles in medical journals and how these systems might be evaluated to support shared decision-making.