Application of Artificial Intelligence to Patient-Targeted Health Information on Kidney Stone Disease

Objective: The American Medical Association recommends health information to be written at a 6th grade level reading level. Our aim was to determine whether Artificial Intelligence can outperform the existing health information on kidney stone prevention and treatment.

Methods: The top 50 search results for "Kidney Stone Prevention" and "Kidney Stone Treatment" on Google, Bing, and Yahoo were selected. Duplicate webpages, advertisements, pages intended for health professionals such as science articles, links to videos, paid subscription pages, and links nonrelated to kidney stone prevention and/or treatment were excluded. Included pages were categorized into academic, hospital-affiliated, commercial, nonprofit foundations, and other. Quality and readability of webpages were evaluated using validated tools, and the reading level was descriptively compared with ChatGPT generated health information on kidney stone prevention and treatment.

Results: 50 webpages on kidney stone prevention and 49 on stone treatment were included in this study. The reading level was determined to equate to that of a 10th to 12th grade student. Quality was measured as "fair" with no pages scoring "excellent" and only 20% receiving a "good" quality. There was no significant difference between pages from academic, hospital-affiliated, commercial, and nonprofit foundation publications. The text generated by ChatGPT was considerably easier to understand with readability levels measured as low as 5th grade.

Conclusions: The language used in existing information on kidney stone disease is of subpar quality and too complex to understand. Machine learning tools could aid in generating information that is comprehensible by the public.