-
PRISM: Patient Records Interpretation for Semantic Clinical Trial Matching using Large Language Models
Authors:
Shashi Kant Gupta,
Aditya Basu,
Mauro Nievas,
Jerrin Thomas,
Nathan Wolfrath,
Adhitya Ramamurthi,
Bradley Taylor,
Anai N. Kothari,
Regina Schwind,
Therica M. Miller,
Sorena Nadaf-Rahrov,
Yanshan Wang,
Hrituraj Singh
Abstract:
Clinical trial matching is the task of identifying trials for which patients may be potentially eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. This process is manual, time-intensive, and challenging to scale up, resulting in many patients miss…
▽ More
Clinical trial matching is the task of identifying trials for which patients may be potentially eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. This process is manual, time-intensive, and challenging to scale up, resulting in many patients missing out on potential therapeutic options. Recent advancements in Large Language Models (LLMs) have made automating patient-trial matching possible, as shown in multiple concurrent research studies. However, the current approaches are confined to constrained, often synthetic datasets that do not adequately mirror the complexities encountered in real-world medical data. In this study, we present the first, end-to-end large-scale empirical evaluation of clinical trial matching using real-world EHRs. Our study showcases the capability of LLMs to accurately match patients with appropriate clinical trials. We perform experiments with proprietary LLMs, including GPT-4 and GPT-3.5, as well as our custom fine-tuned model called OncoLLM and show that OncoLLM, despite its significantly smaller size, not only outperforms GPT-3.5 but also matches the performance of qualified medical doctors. All experiments were carried out on real-world EHRs that include clinical notes and available clinical trials from a single cancer center in the United States.
△ Less
Submitted 26 April, 2024; v1 submitted 23 April, 2024;
originally announced April 2024.
-
Distilling Large Language Models for Matching Patients to Clinical Trials
Authors:
Mauro Nievas,
Aditya Basu,
Yanshan Wang,
Hrituraj Singh
Abstract:
The recent success of large language models (LLMs) has paved the way for their adoption in the high-stakes domain of healthcare. Specifically, the application of LLMs in patient-trial matching, which involves assessing patient eligibility against clinical trial's nuanced inclusion and exclusion criteria, has shown promise. Recent research has shown that GPT-3.5, a widely recognized LLM developed b…
▽ More
The recent success of large language models (LLMs) has paved the way for their adoption in the high-stakes domain of healthcare. Specifically, the application of LLMs in patient-trial matching, which involves assessing patient eligibility against clinical trial's nuanced inclusion and exclusion criteria, has shown promise. Recent research has shown that GPT-3.5, a widely recognized LLM developed by OpenAI, can outperform existing methods with minimal 'variable engineering' by simply comparing clinical trial information against patient summaries. However, there are significant challenges associated with using closed-source proprietary LLMs like GPT-3.5 in practical healthcare applications, such as cost, privacy and reproducibility concerns. To address these issues, this study presents the first systematic examination of the efficacy of both proprietary (GPT-3.5, and GPT-4) and open-source LLMs (LLAMA 7B,13B, and 70B) for the task of patient-trial matching. Employing a multifaceted evaluation framework, we conducted extensive automated and human-centric assessments coupled with a detailed error analysis for each model. To enhance the adaptability of open-source LLMs, we have created a specialized synthetic dataset utilizing GPT-4, enabling effective fine-tuning under constrained data conditions. Our findings reveal that open-source LLMs, when fine-tuned on this limited and synthetic dataset, demonstrate performance parity with their proprietary counterparts. This presents a massive opportunity for their deployment in real-world healthcare applications. To foster further research and applications in this field, we release both the annotated evaluation dataset along with the fine-tuned LLM -- Trial-LLAMA -- for public use.
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
Point Cloud Subsampling Parallelization for Unified Memory Platforms
Authors:
Martin Nievas,
Claudio J. Paz,
Gastón R. Araguás
Abstract:
The exploration of unknown environments using robots is a task that integrates different areas such as location, mapping, and planning. For mapping, there are numerous methods to represent the environments through which a robot can travel, in two and three dimensions. The probabilistic occupation grid, Octomap, and STVL can be mentioned among the most important in recent years. Nowadays, RGB-D cam…
▽ More
The exploration of unknown environments using robots is a task that integrates different areas such as location, mapping, and planning. For mapping, there are numerous methods to represent the environments through which a robot can travel, in two and three dimensions. The probabilistic occupation grid, Octomap, and STVL can be mentioned among the most important in recent years. Nowadays, RGB-D cameras are widely used to generate a detailed representation of the environment. RGB-D camera measurements present a large volume of data, which must be reduced in order to be used in platforms with limited computing resources. This work presents an implementation of the point cloud decimation method capable of being executed on platforms with unified memory. It consists of reducing the point cloud iteratively using a subdivision of space. Results were obtained for different sizes of grids, platforms, and scenarios, both real and simulated. The results indicate that in embedded systems it is convenient to have architectures that share memory between CPU and GPU to optimize data block communication processes.
△ Less
Submitted 22 February, 2021;
originally announced February 2021.
