Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS

Miguel Angrick; Shiyu Luo; Qinwan Rabbani; Daniel N Candrea; Samyak Shah; Griffin W Milsap; William S Anderson; Chad R Gordon; Kathryn R Rosenblatt; Lora Clawson; Nicholas Maragakis; Francesco V Tenore; Matthew S Fifer; Hynek Hermansky; Nick F Ramsey; Nathan E Crone

doi:10.1101/2023.06.30.23291352

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS

medRxiv [Preprint]. 2023 Jul 1:2023.06.30.23291352. doi: 10.1101/2023.06.30.23291352.

Authors

Miguel Angrick¹, Shiyu Luo², Qinwan Rabbani³, Daniel N Candrea², Samyak Shah¹, Griffin W Milsap⁴, William S Anderson⁵, Chad R Gordon^{5

6}, Kathryn R Rosenblatt^{1

7}, Lora Clawson¹, Nicholas Maragakis¹, Francesco V Tenore⁴, Matthew S Fifer⁴, Hynek Hermansky^{8

9}, Nick F Ramsey¹⁰, Nathan E Crone¹

Affiliations

¹ Department of Neurology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
² Department of Biomedical Engineering, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
³ Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, USA.
⁴ Research and Exploratory Development Department, Johns Hopkins Applied Physics Laboratory, Laurel, MD, USA.
⁵ Department of Neurosurgery, The Johns Hopkins University School of Medicine, Baltimore, MD.
⁶ Section of Neuroplastic and Reconstructive Surgery, Department of Plastic Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
⁷ Department of Anesthesiology & Critical Care Medicine, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
⁸ Center for Language and Speech Processing, The Johns Hopkins University, Baltimore, MD, USA.
⁹ Human Language Technology Center of Excellence, The Johns Hopkins University, Baltimore, MD, USA.
¹⁰ UMC Utrecht Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, The Netherlands.

Abstract

Recent studies have shown that speech can be reconstructed and synthesized using only brain activity recorded with intracranial electrodes, but until now this has only been done using retrospective analyses of recordings from able-bodied patients temporarily implanted with electrodes for epilepsy surgery. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a clinical trial participant (ClinicalTrials.gov, NCT03567213) with dysarthria due to amyotrophic lateral sclerosis (ALS). We demonstrate a reliable BCI that synthesizes commands freely chosen and spoken by the user from a vocabulary of 6 keywords originally designed to allow intuitive selection of items on a communication board. Our results show for the first time that a speech-impaired individual with ALS can use a chronically implanted BCI to reliably produce synthesized words that are intelligible to human listeners while preserving the participants voice profile.

Publication types

Preprint

Associated data

ClinicalTrials.gov/NCT03567213

Grants and funding

UH3 NS114439/NS/NINDS NIH HHS/United States