Paralyzed after a stroke for the first time “spoke” through a digital avatar

An almost completely paralyzed woman was able to “speak” through a digital avatar for the first time in the world. The latest technology uses tiny electrodes on the surface of the brain and is faster than synthesizers that rely on eye tracking.

A severely paralyzed woman was able to speak through an avatar using technology that converted her brain signals into speech and facial expressions.

This achievement raises hope that brain-computer interfaces (BCIs) may be on the verge of transforming the lives of people who have lost the ability to speak due to diseases such as strokes and amyotrophic lateral sclerosis (ALS).

Until now, patients have had to rely on frustratingly slow speech synthesizers that involve speaking words using eye tracking or small facial movements, making natural conversation impossible.

The latest technology uses tiny electrodes implanted on the surface of the brain to detect electrical activity in the part of the brain that controls speech and facial movements. These signals are translated directly into speech and facial expressions of the digital avatar, including a smile, frown, or surprise.

“Our goal is to restore a full, embodied way of communicating that is actually the most natural way for us to communicate with others,” said Professor Edward Chang, who led the work at the University of California, San Francisco (UCSF). “These advances bring us a significant step closer to making this a real solution for patients.”

Ann, a 47-year-old patient, was severely paralyzed after suffering a brain stem stroke more than 18 years ago, according to The Guardian. She cannot speak or type, and usually communicates using motion-tracking technology, which allows her to slowly pick up letters at up to 14 words per minute. She hopes that avatar technology will allow her to work as a consultant in the future.

The team implanted a paper-thin rectangle of 253 electrodes on the surface of Ann’s brain in an area critical for speech. The electrodes intercepted brain signals that, if not for the stroke, would have controlled the muscles in her tongue, jaw, larynx, and face.

After being implanted, Ann worked with the team to train the system’s artificial intelligence algorithm to recognize her brain’s unique signals for various speech sounds by repeating different phrases over and over again.

The computer learned 39 distinctive sounds, and a GPT-chat style language model was used to translate the signals into understandable sentences. This was then used to control an avatar with a voice tuned to sound like Ann’s voice before her injury, based on a recording of her performing at her wedding.

The technology was flawed, in a test run of more than 500 phrases, words were spelled wrong 28% of the time, and it generated “brain text” at a rate of 78 words per minute, compared to the 110-150 words normally spoken in natural conversation. notes The Guardian.

However, the scientists said recent advances in accuracy, speed and sophistication indicate that the technology is now at the stage of practical utility for patients.

Professor Nick Ramsey, a neuroscientist at the University of Utrecht in the Netherlands, who was not involved in the study, said: “This is a significant jump from previous results. We are at a turning point.”

The next important step is to create a wireless version of the brain-computer interfaces that could be implanted under the skull. “Giving people the freedom to control their own computers and phones with this technology would have a profound impact on their independence and social interactions,” said Dr. David Moses, assistant professor of neurological surgery at the University of California and co-author of the study. The results of the study are published in the journal Nature.