Benchmarking State-of-the-Art Large Language Models for Migraine Patient Education: Performance Comparison of Responses to Common Queries

Linger Li; Pengfei Li; Kun Wang; Liang Zhang; Hongwei Ji; Hongqin Zhao

doi:10.2196/55927

Benchmarking State-of-the-Art Large Language Models for Migraine Patient Education: Performance Comparison of Responses to Common Queries

J Med Internet Res. 2024 Jul 23:26:e55927. doi: 10.2196/55927.

Authors

Linger Li¹, Pengfei Li², Kun Wang¹, Liang Zhang¹, Hongwei Ji^#^{3

4}, Hongqin Zhao^#¹

Affiliations

¹ Department of Neurology, The Affiliated Hospital of Qingdao University, Qingdao, China.
² Department of Internal Medicine, The Affiliated Hospital of Qingdao University, Qingdao, China.
³ Tsinghua Medicine, Tsinghua University, Beijing, China.
⁴ Department of Internal Medicine, Beijing Tsinghua Changgung Hospital, Beijing, China.

^# Contributed equally.

PMID: 38828692
PMCID: PMC11303883
DOI: 10.2196/55927

Abstract

This study assessed the potential of large language models (OpenAI's ChatGPT 3.5 and 4.0, Google Bard, Meta Llama2, and Anthropic Claude2) in addressing 30 common migraine-related queries, providing a foundation to advance artificial intelligence-assisted patient education and insights for a holistic approach to migraine management.

Keywords: AI; AI-assisted; ChatGPT; Google Bard; OpenAI; accuracy; artificial intelligence; education; headache; holistic; language model; large language models; management; migraine; migraine management; patient education.

©Linger Li, Pengfei Li, Kun Wang, Liang Zhang, Hongwei Ji, Hongqin Zhao. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 23.07.2024.

Publication types

Comparative Study

MeSH terms

Artificial Intelligence
Benchmarking*
Humans
Migraine Disorders* / therapy
Patient Education as Topic* / methods