Using voice recognition and machine learning techniques for detecting patient-reported outcomes from conversational voice in palliative care patients

Aim: Patient-reported outcome measures (PROMs) are increasingly used in palliative care to evaluate patients' symptoms and conditions. Healthcare providers often collect PROMs through conversations. However, the manual entry of these data into electronic medical records can be burdensome for healthcare providers. Voice recognition technology has been explored as a potential solution for alleviating this burden. However, research on voice recognition technology for palliative care is lacking. This study aimed to verify the use of voice recognition and machine learning to automatically evaluate PROMs using clinical conversation voice data.

Methods: We recruited 100 home-based palliative care patients from February to May 2023, conducted interviews using the Integrated Palliative Care Outcome Scale (IPOS), and transcribed their voice data using an existing voice recognition tool. We calculated the recognition rate and developed a machine learning model for symptom detection. Model performance was primarily evaluated using the F1 score, harmonic mean of the model's positive predictive value, and recall.

Results: The mean age of the patients was 80.6 years (SD, 10.8 years), and 34.0% were men. Thirteen patients had cancer, and 87 did not. The patient voice recognition rate of 55.6% (SD, 12.1%) was significantly lower than the overall recognition rate of 76.1% (SD, 6.4%). The F1 scores for the five total symptoms ranged from 0.31 to 0.46.

Conclusion: Although further improvements are necessary to enhance our model's performance, this study provides valuable insights into voice recognition and machine learning in clinical settings. We expect our findings will reduce the burden of recording PROMs on healthcare providers, increasing the wider use of PROMs.