Evaluation and Real-World Performance Monitoring of Artificial Intelligence Models in Clinical Practice: Try It, Buy It, Check It

J Am Coll Radiol. 2021 Nov;18(11):1489-1496. doi: 10.1016/j.jacr.2021.08.022. Epub 2021 Sep 30.

Abstract

The pace of regulatory clearance of artificial intelligence (AI) algorithms for radiology continues to accelerate, and numerous algorithms are becoming available for use in clinical practice. End users of AI in radiology should be aware that AI algorithms may not work as expected when used beyond the institutions in which they were trained, and model performance may degrade over time. In this article, we discuss why regulatory clearance alone may not be enough to ensure AI will be safe and effective in all radiological practices and review strategies available resources for evaluating before clinical use and monitoring performance of AI models to ensure efficacy and patient safety.

Keywords: AI evaluation; AI purchase; AI regulation.

MeSH terms

  • Algorithms
  • Artificial Intelligence*
  • Humans
  • Radiography
  • Radiology*