Published In Nature: New Breakthrough of Speech-to-Text BCI Achieves Speed of 62 Words/Minute

Speech brain–computer interfaces (BCIs) is a innovate technology that establishes a communication channel between a user and certain devices via the brain signal of the users, they have great potential for restoring speech and communication capabilities in disabled patients.

Early studies, although promising, have not yet achieved the sufficient high accuracies for decoding brain activities to facilitate communication of unconstrained sentences from a large vocabulary.

To bridge this gap, in a new paper A high-performance speech neuroprosthesis, a research team from Stanford University, Washington University in St. Louis, VA RR&D Center for Neurorestoration and Neurotechnology, Brown University, Harvard Medical School presents a high-performance speech-to-text BCI that is able to process unconstrained sentences from a large vocabulary at a speed of 62 words/minute, which substantially exceeds the communication rates of conventional technologies for people with paralysis.

The team first investigates how orofacial movement and speech production are organized in motor cortex by recording neural activities in the BrainGate2 pilot clinical trial, they found a strong tuning of all categories of tested movements in
area 6v.

Next, they explores how information was distributed across area 6v for each movement, they found that the speech decode rate is more accurate from ventral array, whereas the dorsal array carry more orofacial movements information. Nevertheless, 6v arrays contain rich information of all movement categories. Finally, all speech articulators can be clearly represented within 3.2 × 3.2 mm^2 arrays.

Moving forward, the researchers tested whether they can neutrally decode whole sentences in real time. They leverage custom machine learning approaches adapted from modern speech recognition to train a recurrent neural network (RNN), which achieves high performance on limited amount of neural data.

By leveraging all of their observations, the proposed approach achieve 92% decoding accuracy rate decode from among 33 orofacial movements, 62% accuracy among 39 phonemes and 94% accuracy of 50 words. Moreover, the speech-to-text BCI achieves speed of 62 words per minute.

The results verify that the proposed speech BCIs move steps forward for restoring rapid communication to people who has difficulty in speaking. And to the researchers’ best knowledge, this is the first time that speech BCIs significantly exceed the communication rates of all existing technologies for people with paralysis.

The paper A high-performance speech neuroprosthesis on Nature.

---

Author: Hecate He | Editor: Chain Zhang

---

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.