Quality of interaction between clinicians and artificial intelligence systems. A systematic review

Introduction: Artificial intelligence (AI) has the potential to improve healthcare quality when thoughtfully integrated into clinical practice. Current evaluations of AI solutions tend to focus solely on model performance. There is a critical knowledge gap in the assessment of AI-clinician interactions. We systematically reviewed existing literature to identify interaction traits that can be used to assess the quality of AI-clinician interactions.

Methods: We performed a systematic review of published studies to June 2022 that reported elements of interactions that impacted the relationship between clinicians and AI-enabled clinical decision support systems. Due to study heterogeneity, we conducted a narrative synthesis of the different interaction traits identified from this review. Two study authors categorised the AI-clinician interaction traits based on their shared constructs independently. After the independent categorisation, both authors engaged in a discussion to finalise the categories.

Results: From 34 included studies, we identified 210 interaction traits. The most common interaction traits included usefulness, ease of use, trust, satisfaction, willingness to use and usability. After removing duplicate or redundant traits, 90 unique interaction traits were identified. Unique interaction traits were then classified into seven categories: usability and user experience, system performance, clinician trust and acceptance, impact on patient care, communication, ethical and professional concerns, and clinician engagement and workflow.

Discussion: We identified seven categories of interaction traits between clinicians and AI systems. The proposed categories may serve as a foundation for a framework assessing the quality of AI-clinician interactions.