Scientists trained an AI model that translates brain activity into sound to help a stroke survivor regain speech.
ADVERTISEMENT
Scientists have developed a device that can translate thoughts about speech into spoken words in real time.
Although it’s still experimental, they hope the brain-computer interface could someday help give voice to those unable to speak.
A new study described testing the device on a 47-year-old woman, named Ann,** with quadriplegia who couldn’t speak for 18 years after a stroke. Doctors implanted it in her brain during surgery as part of a clinical trial.
It "listens to her intent to make movements of the tongue and the jaw," Gopala Anumanchipalli, a co-author of the study, which was published in the journal Nature Neuroscience, told Euronews Health.
The implant itself sits on the speech centre of the brain so that it’s listening in, and those signals are translated into pieces of speech that make up sentences.
Other brain-computer interfaces (BCIs) for speech typically have a slight delay between thoughts of sentences and computerised verbalisation. Such delays can disrupt the natural flow of conversation, potentially leading to miscommunication and frustration, researchers said.
This is "a pretty big advance in our field," said Jonathan Brumberg of the Speech and Applied Neuroscience Lab at the University of Kansas in the US. He was not part of the study.
How the implant works
A team in California recorded the woman’s brain activity using electrodes while she spoke sentences silently in her brain.
The scientists used a synthesiser they had built using her wedding video shot before her injury to create a speech sound that she would have spoken. Then they trained an artificial intelligence (AI) model that translates neural activity into units of sound.
Previously, Ann was using an augmentative and alternative communication system (AAC) to communicate and write emails.
"The eye-tracking technology is really rudimentary. It is very error-prone and it's really boring," Anumanchipalli said.
"We are trying to like rehabilitate as much of that naturalistic behaviour as possible for Anne. For someone who is paralysed, it's extremely important to be able to communicate with minimal delay... so that they can let their thoughts known and communicate with the world," he added.
"So this is a really big deal for someone who has lost speaking ability".
Reconnect them to the world
The new tech works similarly to existing systems used to transcribe meetings or phone calls in real time, said Anumanchipalli, of the University of California, Berkeley, in the US.
It’s a "streaming approach," Anumanchipalli said, with each 80-millisecond chunk of speech – about half a syllable – sent into a recorder.
Decoding speech that quickly has the potential to keep up with the fast pace of natural speech, said Brumberg.
Researchers say it allows embodiment, a sense of autonomy that the BCI feels like an extension of one's own body just like when we learn to bike or swim.
"Once you get the hang of it, it's like you are doing it, right? It is that kind of embodiment, that feeling of 'hey, that's me', that really is important in creating brain computer interfaces," Anumanchipalli told Euronews Health.
He says the real-time tech can help improve the quality of life of paralysed patients, who suffer "locked-in syndrome".
"If you should track the quality of life, this drop in the quality of life happens when they are not able to communicate. So it is very impactful to be able to reconnect them to the world so that they're not locked in," he added.
The use of voice samples, he added, "would be a significant advance in the naturalness of speech".
Though the work was partially funded by the US National Institutes of Health (NIH), Anumanchipalli said it wasn’t affected by recent NIH research cuts.
More research is needed before the technology is ready for wide use, but with "sustained investments," it could be available to patients within a decade, he said.
