The Definition of Glaucomatous Optic Neuropathy in Artificial Intelligence Research and Clinical Applications

Objective: Although artificial intelligence (AI) models may offer innovative and powerful ways to use the wealth of data generated by diagnostic tools, there are important challenges related to their development and validation. Most notable is the lack of a perfect reference standard for glaucomatous optic neuropathy (GON). Because AI models are trained to predict presence of glaucoma or its progression, they generally rely on a reference standard that is used to train the model and assess its validity. If an improper reference standard is used, the model may be trained to detect or predict something that has little or no clinical value. This article summarizes the issues and discussions related to the definition of GON in AI applications as presented by the Glaucoma Workgroup from the Collaborative Community for Ophthalmic Imaging (CCOI) US Food and Drug Administration Virtual Workshop, on September 3 and 4, 2020, and on January 28, 2022.

Design: Review and conference proceedings.

Subjects: No human or animal subjects or data therefrom were used in the production of this article.

Methods: A summary of the Workshop was produced with input and approval from all participants.

Main outcome measures: Consensus position of the CCOI Workgroup on the challenges in defining GON and possible solutions.

Results: The Workshop reviewed existing challenges that arise from the use of subjective definitions of GON and highlighted the need for a more objective approach to characterize GON that could facilitate replication and comparability of AI studies and allow for better clinical validation of proposed AI tools. Different tests and combination of parameters for defining a reference standard for GON have been proposed. Different reference standards may need to be considered depending on the scenario in which the AI models are going to be applied, such as community-based or opportunistic screening versus detection or monitoring of glaucoma in tertiary care.

Conclusions: The development and validation of new AI-based diagnostic tests should be based on rigorous methodology with clear determination of how the reference standards for glaucomatous damage are constructed and the settings where the tests are going to be applied.

Financial disclosure(s): Proprietary or commercial disclosure may be found after the references.