Establishing the Foundations of Emotional Intelligence in Care Companion Robots to Mitigate Agitation Among High-Risk Patients With Dementia: Protocol for an Empathetic Patient-Robot Interaction Study

Background: An estimated 6.7 million persons are living with dementia in the United States, a number expected to double by 2060. Persons experiencing moderate to severe dementia are 4 to 5 times more likely to fall than those without dementia, due to agitation and unsteady gait. Socially assistive robots fail to address the changing emotional states associated with agitation, and it is unclear how emotional states change, how they impact agitation and gait over time, and how social robots can best respond by showing empathy.

Objective: This study aims to design and validate a foundational model of emotional intelligence for empathetic patient-robot interaction that mitigates agitation among those at the highest risk: persons experiencing moderate to severe dementia.

Methods: A design science approach will be adopted to (1) collect and store granular, personal, and chronological data using Personicle (an open-source software platform developed to automatically collect data from phones and other devices), incorporating real-time visual, audio, and physiological sensing technologies in a simulation laboratory and at board and care facilities; (2) develop statistical models to understand and forecast the emotional state, agitation level, and gait pattern of persons experiencing moderate to severe dementia in real time using machine learning and artificial intelligence and Personicle; (3) design and test an empathy-focused conversation model, focused on storytelling; and (4) test and evaluate this model for a care companion robot (CCR) in the community.

Results: The study was funded in October 2023. For aim 1, architecture development for Personicle data collection began with a search for existing open-source data in January 2024. A community advisory board was formed and met in December 2023 to provide feedback on the use of CCRs and provide personal stories. Full institutional review board approval was received in March 2024 to place cameras and CCRs at the sites. In March 2024, atomic marker development was begun. For aim 2, after a review of open-source data on patients with dementia, the development of an emotional classifier was begun. Data labeling was started in April 2024 and completed in June 2024 with ongoing validation. Moreover, the team established a baseline multimodal model trained and validated on healthy-person data sets, using transformer architecture in a semisupervised manner, and later retrained on the labeled data set of patients experiencing moderate to severe dementia. In April 2024, empathy alignment of large language models was initiated using prompt engineering and reinforcement learning.

Conclusions: This innovative caregiving approach is designed to recognize the signs of agitation and, upon recognition, intervene with empathetic verbal communication. This proposal has the potential to have a significant impact on an emerging field of computational dementia science by reducing unnecessary agitation and falls of persons experiencing moderate to severe dementia, while reducing caregiver burden.

International registered report identifier (irrid): PRR1-10.2196/55761.