ChatGPT-4: Alcohol use disorder responses

Background and aims: Alcohol use disorder (AUD) is characterized by low levels of engagement with effective treatments. Enhancing awareness of AUD treatments and how to navigate the treatment system is crucial. Many individuals use online sources (e.g. search engines) for answers to health-related questions; web-based results include a mix of high- and low-quality information. Artificial intelligence may improve access to quality health information by providing concise, high-quality responses to complex health-related questions. This study evaluated the quality of ChatGPT-4 responses to AUD-related queries.

Method: A comprehensive list of 64 AUD-related questions was developed through a combination of Google Trends analysis and expert consultation. ChatGPT-4 was prompted with each question, followed by a request to provide 3-5 peer-reviewed scientific citations supporting each response. Responses were evaluated for whether they were evidence-based, provided a referral and provided supporting documentation.

Results: ChatGPT-4 responded to all AUD-related queries, with 92.2% (59/64) of responses being fully evidence-based. Although only 12.5% (8/64) of responses included referrals to external resources, all responses (100%; 5/5) to location-specific ('near me') queries directed individuals to appropriate resources like the NIAAA Treatment Navigator. Most (85.9%; 55/64) responses to the follow-up question provided supporting documentation.

Conclusions: ChatGPT-4 responds to alcohol use disorder-related questions with evidence-based information and supporting documentation. ChatGPT-4 could be promoted as a reasonable resource for those looking online for alcohol use disorder-related information.