Feasibility Analysis of Phenotype Quantification from Unstructured Clinical Interactions

Daniel S Barron; Stephen Heisig; Carla Agurto; Raquel Norel; Brittany Quagan; Albert Powers; Michael L Birnbaum; Todd Constable; Guillermo Cecchi; John H Krystal

doi:10.5334/cpsy.78

Feasibility Analysis of Phenotype Quantification from Unstructured Clinical Interactions

Comput Psychiatr. 2022 Jan 11;6(1):1-7. doi: 10.5334/cpsy.78. eCollection 2022.

Authors

Daniel S Barron^{1

2

3}, Stephen Heisig⁴, Carla Agurto⁵, Raquel Norel⁵, Brittany Quagan¹, Albert Powers¹, Michael L Birnbaum⁶, Todd Constable^{7

8}, Guillermo Cecchi⁵, John H Krystal¹

Affiliations

¹ Department of Psychiatry, Yale University, New Haven, CT, USA.
² Department of Psychiatry, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
³ Department of Anesthesiology and Pain Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
⁴ Department of Neurology, Icahn School of Medicine, Mt. Sinai, NY, USA.
⁵ T.J. Watson IBM Research Laboratory, Yorktown Heights, NY, USA.
⁶ Department of Psychiatric Research, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, USA.
⁷ Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, USA.
⁸ Department of Neurosurgery, Yale School of Medicine, New Haven, CT 06520, USA.

Abstract

We conducted a feasibility analysis to determine the quality of data that could be collected ambiently during routine clinical conversations. We used inexpensive, consumer-grade hardware to record unstructured dialogue and open-source software tools to quantify and model face, voice (acoustic and language) and movement features. We used an external validation set to perform proof-of-concept predictive analyses and show that clinically relevant measures can be produced without a restrictive protocol.

Keywords: acoustic; conversation; digital phenotype; facial feature; voice.

Grants and funding

DSB was partially funded by the National Institute of Mental Health Grant No. 5 T32 MH 19961-22. The authors are thankful to the volunteer participants without whose active involvement, the present study would not have been possible.