A team of researchers from the University of California in San Francisco (UCSF) in the United States developed a brain implant capable of simultaneously decoding people's speech and gestures and reproducing them on a screen.
© Shutterstock
15/09/2026
by
Lusa
So, observed that a single implant could capture signals related to various types of movements, including upper limbs and mouth and face movements related to speech (like shrug, lift your fist or wave your head).
The authors used parallel speech decoders and gestures to animate a whole-body avatar with two of the patients.
The difference between these decoders is that they translate the thoughts of the participants into commands that determine the words and expressions of a human avatar on a screen.
To prove their effectiveness, they asked participants to try to make specific gestures, respond to conversational indications or do both simultaneously.
In the conversational tasks, the system accurately decoded the speech and gestures of both participants and one of them achieved an average accuracy of 100%, both in the decoding of speech and in the gestures, along three blocks of themes.
The authors found that training the system using speech and gestures together improved performance and reduced errors.
“While testing with more participants and with broader sets of words and gestures are needed before technology can be widely used, we believe that our device can enable multifaceted and realistic communication“, highlighted one of the authors, Edward Chang, professor of neurosurgery at UCSF.
According to Chang, the work opens the possibility of restoring the natural dynamism of communication in people who have lost their verbal and non-verbal abilities.
Although the device used in this study consists of a wired system that connects sensors deployed to external processing units, Chang advanced in a statement that his team will soon test a fully implantable and wireless version with better long-term application prospects.