Success of ChatGPT, an AI language model, in taking the French language version of the European Board of Ophthalmology examination: A novel approach to medical knowledge assessment

C Panthier; D Gatinel

doi:10.1016/j.jfo.2023.05.006

Success of ChatGPT, an AI language model, in taking the French language version of the European Board of Ophthalmology examination: A novel approach to medical knowledge assessment

J Fr Ophtalmol. 2023 Sep;46(7):706-711. doi: 10.1016/j.jfo.2023.05.006. Epub 2023 Aug 1.

Authors

C Panthier¹, D Gatinel²

Affiliations

¹ Department of Ophthalmology, Rothschild Foundation Hospital, 25, rue Manin, 75019 Paris, France; Center of Expertise and Research in Optics for Vision (CEROV), Paris, France.
² Department of Ophthalmology, Rothschild Foundation Hospital, 25, rue Manin, 75019 Paris, France; Center of Expertise and Research in Optics for Vision (CEROV), Paris, France. Electronic address: [email protected].

PMID: 37537126
DOI: 10.1016/j.jfo.2023.05.006

Abstract

Purpose: The purpose of this study was to evaluate the performance of ChatGPT, a cutting-edge artificial intelligence (AI) language model developed by OpenAI, in successfully completing the French language version of the European Board of Ophthalmology (EBO) examination and to assess its potential role in medical education and knowledge assessment.

Methods: ChatGPT, based on the GPT-4 architecture, was exposed to a series of EBO examination questions in French, covering various aspects of ophthalmology. The AI's performance was evaluated by comparing its responses with the correct answers provided by ophthalmology experts. Additionally, the study assessed the time taken by ChatGPT to answer each question as a measure of efficiency.

Results: ChatGPT achieved a 91% success rate on the EBO examination, demonstrating a high level of competency in ophthalmology knowledge and application. The AI provided correct answers across all question categories, indicating a strong understanding of basic sciences, clinical knowledge, and clinical management. The AI model also answered the questions rapidly, taking only a fraction of the time needed by human test-takers.

Conclusion: ChatGPT's performance on the French language version of the EBO examination demonstrates its potential to be a valuable tool in medical education and knowledge assessment. Further research is needed to explore optimal ways to implement AI language models in medical education and to address the associated ethical and practical concerns.

Keywords: AI applications; Apprentissage par la machine; Apprentissage profond; Artificial intelligence; ChatGPT; Deep learning; Entraînement sur base de données; Ethics in AI; European Board of Ophthalmology; Examen médical; Generative AI; Générateur de texte; Génération d’IA; Human-like interaction; Intelligence artificielle; Language model; Machine learning; Medical examination; Modèle conversationnel; Natural language processing; OpenAI; Ophtalmologie; Ophthalmology; Simulation d’interaction humaine; Text generation; Training dataset; Transformateur d’architecture; Transformer architecture; Éthique en IA.

MeSH terms

Artificial Intelligence*
Humans
Language
Ophthalmology*