Brain implants are helping people who’ve lost their voice

On paper, my interview with Casey Harrell looks like any other.

I ask the questions, he answers them. He tells me about his background as a climate activist, his Southern roots and his daughter, who as a preschooler loved to dress up like a cheetah and roar around the house. “My daughter was incredibly cute in her cheetah outfit,” Harrell says.

The transcript tells the story of our conversation. But between the lines is an entirely different story — the story of where Harrell’s words come from.

Harrell started losing his ability to speak in his early 40s, nearly six years ago. He talked to me using a brain implant that can decode his words directly from his thoughts. The device, called a brain-computer interface, is experimental. But it has transformed Harrell’s life. And it could soon transform others. Advances in computing power and AI, lessons gleaned from patient experience and investments from neurotechnology companies are starting to yield real progress on brain-computer interfaces for people who can no longer talk. In other words, Harrell’s new voice might not be so singular in the future.

“There’s a lot of excitement and hope in the field right now,” says Matthew Willsey, a neuro­surgeon and biomedical engineer at the University of Michigan in Ann Arbor, who recently implanted a new type of speech interface in a woman’s brain. For years, Willsey says, scientists didn’t know whether attempts to decode speech would be successful; now, a body of research results points clearly to yes. “They’re thinking, ‘Wow, this is actually a usable technology that addresses a real problem that people suffer from.’”

Harrell has amyotrophic lateral sclerosis, or ALS. The progressive neurodegenerative disease, which affects an estimated 35,000 people in the United States, kills nerve cells in the brain and spinal cord that control muscles. As the neural connections between Harrell’s brain and body began to falter, he gradually lost his ability to move his muscles, leaving him confined to his bed or a wheelchair. Some of those dropped signals directed the muscles in his throat and mouth, the ones that allow him to speak.

“I began to lose my ability to talk shortly after my diagnosis in 2020,” Harrell says. After that, it was “a slow-motion car crash” as fewer and fewer people could understand him. “[My voice] is the most important thing I have lost in this whole disease,” he says.

In a stroke of luck, Harrell learned through a family member about a clinical trial happening at the University of California, Davis — one that attempted to restore speech with a brain-computer interface. In 2023, Harrell had four small devices implanted on the left side of his brain. These grids of electrodes reside on Harrell’s speech motor cortex, a stretch of brain along the precentral gyrus that controls the movements required for speech.

Today, Harrell uses his system for about 10 to 14 hours daily, making him, as he puts it, “the world’s biggest power user of the brain-to-computer interface known to man.” Hard data suggest he’s right. A recent count shows that Harrell has used his system to communicate nearly 2 million words, the scientists working with him reported in June in Nature Medicine.

“I began to lose my ability to talk shortly after my diagnosis in 2020 … [My voice] is the most important thing I have lost in this whole disease.”

Casey Harrell

The very existence of Harrell’s words shows it may be possible to restore speech for people with ALS, as well as for those whose speech is impaired by stroke, paralysis, cerebral palsy and Parkinson’s disease. “This kind of technology has the potential to dramatically change the conversation,” says David Brandman, a UC Davis neurosurgeon and neuroscientist who works with Harrell.

Harrell’s interface still has room for improvement. It’s bulky, relatively slow and limited in what sorts of nonword sounds it can decode.

But it is a “beginning,” says Leigh Hochberg, a neuroscientist at Brown University who works with Harrell. “It’s still early days. There’s a lot of research to happen, and it’s going to be very exciting.” The end game, Hochberg says, is a world in which people could use a device to communicate seamlessly, quickly, clearly — and in their former voices.

Success would mean restoring more than just people’s words, Brandman says. It would mean restoring their ability to connect in all the ineffable ways that make us human. “Speech is so fundamental to who we are,” he says. “Imagine living in a world where you couldn’t tell someone you’re in pain, or you couldn’t say that you love them … It’s not hard to imagine how devastating that would be for your quality of life.”

The voice inside

On the path to a technology that enables fast, natural speech, scientists first had to answer a more basic question: Could the brain’s electrical chatter be decoded and translated into action outside of the head?

Nerve cells speak to each other with tiny pulses of electricity, signals that can be picked up with electrodes. The existence of this electrical language raised the tantalizing idea that electrical messages could be decoded straight from where they originate in the brain, bypassing damaged links to the rest of the body.

“This has really started as early as the 1950s, the first time scientists put electrodes into brains to record neural activity,” says Sergey Stavisky, a neuroscientist and neural engineer at UC Davis. “You can record the electrical activity of brain cells or parts of the brain. And that’s really when modern neuroscience started trying to study, how does the brain compute? How does it allow us to move our muscles?”

Figuring out how to decode simple neural commands was the first step. Animal studies paved the way, including experiments in which rhesus monkeys consciously controlled the movements of a robotic arm using only brain signals detected by implanted electrodes.

By 2004, scientists had enough confidence in the approach to launch a clinical trial in humans called BrainGate. Researchers implanted a grid of electrodes in the brain of a paralyzed man named Matthew Nagle. A former star athlete, Nagle was able to control a computer cursor with his thoughts alone. “That was really the starting gun of the modern version of the brain-computer interface field,” Stavisky says.

From there, attempts to link brain activity with some physical outcome — such as moving a computer mouse, a wheelchair or a robotic arm — blossomed. Some of these efforts focused on the brain signals and regions that controlled movements of the hands and arms. Others focused on a very tricky version of the same basic problem: talking.

A brain-computer interface that could detect the intention to say a word and bypass the muscles needed to say it would be a powerful workaround for people with diseases like ALS or brain injuries. And it is no small thing to people who can no longer talk. “Losing their speech is way more devastating than losing their mobility,” says Jessica Pietrowski, a speech and language pathologist at UC Davis who has worked with many such patients. “Speech is how they maintain their identity in their relationships.”

Harrell emphasizes this point. “What does my voice mean to me? It means so much,” he says. “It is the ability to parent my daughter. It is the ability to connect with my wife. It is the ability to make a wry joke with my friends.”

Looking for a cheetah

Talking feels effortless to most of us, but just saying hello involves a sweeping array of actions — and upwards of 100 muscles. “It’s incredibly complex and intricate, and really, an incredible human feat,” Pietrowski says. 

Consider the “h” sound in “hello.” Known as a glottal fricative, it involves an orchestra of tiny movements. First, the diaphragm contracts to fill the lungs with air. Chest and abdominal muscles push air up through relaxed vocal cords and surrounding tissues. This puff of air then travels to the mouth, where lips, jaw and tongue execute complex sequences of movements that create the rest of the sound. Finally, air waves, perfectly shaped, ripple from the mouth. Then, and only then, “you have speech,” Pietrowski says.

But the beginnings of this nearly miraculous transformation start in the brain, where tiny electrical nerve pulses command the body to speak. In Harrell’s trial, he would attempt to speak words, prompting his motor cortex to send signals to his body. The implanted electrodes would detect the patterns and send them to a computer to interpret. With this work-around, it didn’t matter that Harrell’s muscles didn’t get the message; the computer would.

After Harrell recovered from surgery, researchers began testing the system in his home. They collected his neural data and streamed it to computers on a nearby cart, where machine learning algorithms attempted to match Harrell’s brain activity patterns to those that predictably occur with certain sounds. Called phonemes, these sound units are the building blocks of words. The algorithm assembled the phonemes into words, as it made its best guess about what Harrell was trying to say. To limit the computational power required, the algorithm used a vocabulary of just 50 words.

When the system was plugged in and ready to go, scientists asked Harrell to try and say a specific, random sentence built from that library: “What good is that?”

Harrell got to work. His eyebrows shot up as he saw the words appear on a screen in front of him in response to his thoughts. The decoder worked.

Everyone paused. Some of the researchers started to cry. Harrell and his family became emotional. Being in the room when the switch was flipped on was “incredible,” Brandman says. “The majority of my adult life was working up to that moment, and to see it come to fruition was an incredible experience. But more than that was seeing what impact it immediately had for Casey and his family.”

“Imagine living in a world where you couldn’t tell someone you’re in pain, or you couldn’t say that you love them … It’s not hard to imagine how devastating that would be for your quality of life.”

David Brandman
UC Davis neurosurgeon and neuroscientist

Harrell remembers his feelings that day, too: “Pure joy.” Even though he wasn’t choosing his words for that first test, “it was a profound moment,” he says. “The possibilities for the near future for me and the rest of humanity were overwhelming to realize.”

The second day of testing was even more profound, but in a personal way. This time, the system’s algorithm was using a vocabulary of over 125,000 words. This would be more challenging for the decoder, but it would give Harrell more autonomy to say what he wanted.

Harrell started the session with a joke, Hochberg recalls, along the lines of, “Is this thing on?” But his next line was a real puzzler: “I’m looking for a cheetah.”

Hochberg, who had joined remotely on a video call, was sure it was a mistake. “There’s no way that the second sentence on the second day of using a brain-to-computer interface is going to say anything about a cheetah,” Hochberg says. “That’s obviously a decoding error.”

But there was something the scientist couldn’t see. Harrell’s 4-year-old daughter had bounded into the room, dressed as a cheetah.

The discovery was big scientifically, but it was bigger for what it represented to Harrell. Brain technology could restore not just his words, but also the spontaneous exchanges that had once made up his everyday life.

“It was so amazing to be able to be understood right from the get-go and to be able to talk and respond to my daughter,” Harrell says, “especially because that was lacking completely from my life.”

Hochberg says moments like that represent the promise that brain-computer interfaces hold. “To me, seeing it work that quickly and that powerfully really, really emphasized that these are technologies that are going to be able to restore not only activities of daily living,” he says, “but they’re going to restore communication between parent and child.”

Harrell uses his interface to send email and Slack messages and play Wordle with his wife and child. He uses it to work; in fact, he’s just getting started on a new climate consultancy project. Harrell’s interface has connected him to the world in a way that would have been impossible even a few years ago.

But the system is still a research device. And like any early design, it comes with limitations. First, it isn’t portable; Harrell has wires coming from a port that sits on the top of his head, and he can use it only at home. And it is slow by the standards of typical conversation. You can’t tell from the written transcript of our interview, but our conversations always came with built-in delays. 

For example, as we chatted, we discovered that we had both worked in the same part of Washington, D.C. I asked him if he remembered his old address. The answer came some two minutes later.

“I make a very simple analogy where I talk about having the brain-to-computer interface being akin to trying to catch a train, running to get the last car before it leaves the station,” Harrell says. With the current interface, “if people are patient and wait for my sentence correction, I can [catch the train and] be a part of the conversation.”

Harrell’s system translates his neural activity into words on a screen. If he chooses to do so, he can then use a computerized voice to say the words out loud. “It is brain-to-text, and then it gets translated to the voice,” Harrell says. A brain-to-voice system, in which a person’s thoughts are translated directly into spoken words, he adds, “is the holy grail of what this will become.”

More than words

Harrell and the scientists working with him are getting closer to this very type of real-time communication. Last year in Nature, they debuted a new algorithm that attempted to turn Harrell’s brain signals straight into sounds. The results: He could speak his mind with a delay of just 10 milliseconds. That’s very close to a real-time conversation. But the sounds Harrell made were hard to understand, with just over half of them being intelligible to listeners.

The next generation of devices aims to do better. Willsey, the University of Michigan neuro­surgeon, is working on one such device, a fully implantable array. In June, he implanted what’s called the “cortical module,” about the size of a dime, in the brain of a 68-year-old woman. Like Harrell, the woman could no longer talk in a way that most people could understand.

Unlike the BrainGate system, this new device — made by Austin, Texas–based neurotechnology company Paradromics — is portable. Surgeons implanted it in a stretch of brain tissue that made up the woman’s speech motor cortex, along the precentral gyrus. The cortical module collects neural data and sends it to another device implanted in the chest called a transceiver. From there, signals shoot to a receiver on her chest, where the data gets transformed into sounds — spoken words that play out of speakers on a nearby computer.

Paradromics’ priority is to test whether the implant is safe. If so, scientists can turn to the goal of restoring speech in real time. “The moment that you’re thinking the words, you don’t have to wait for the sentence to complete,” says Willsey, who, like Brandman, is a site investigator for the Paradromics clinical trial. “You can let the words kind of flow out as they would in a normal conversation.”

Scientists also want to decode more than words. Qualities of voice, such as pitch, are important for marking questions, says Vikash Gilja, a Paradromics neuroscientist. Speakers can also alter or pace their conversation to show sarcasm or concern. These sounds are all made by moving muscles, Gilja says, and can therefore be decoded from the brain.

“Our near-term goal right now is just to restore speech,” says Paradromics neurosurgeon Stephen Ryu. “But eventually, we’re hoping that we can actually restore someone’s ability to be fully involved in a conversation, to basically decode things like their timing, their emotions, their personality.”

Other researchers are trying to incorporate the gestures that many of us use to communicate.

“Some gestures can substitute for speech entirely, like nodding yes, shaking your head no, waving, shrugging, giving a thumbs up,” says bioengineer Samantha Brosler of the University of California, San Fransico. Gestures can also change the meaning of words; a “maybe” with a head nod is close to a yes, for instance. “I think of gestures and body movements as an additional communication channel that can work together with speech,” Brosler says.

In a new study, Brosler and colleagues enabled two people with paralysis who had lost their speaking ability to instead use their brain signals to control both the words and gestures of digital avatars. The advance, reported in September in Nature Neuroscience, adds to the repertoire of signals that could be decoded and turned into action. 

And as all this is happening, scientists are creating smaller, more stable types of electrodes to embed in the brain; more powerful computational algorithms to handle larger amounts of data; and better models of how those signals relate to sounds, words and gestures, Gilja says. “We can see all the pieces coming together.”

“It was so amazing to be able to be understood right from the get-go and to be able to talk and respond to my daughter, especially because that was lacking completely from my life.”

Casey Harrell

There are many reasons to be optimistic about the future, says Brandman, who is an investigator for the clinical trial led by Paradromics. “I think I’m about five years away from being able to meet a patient with ALS and having the following conversation,” he says. “‘Ms. Smith, I’m sorry you’ve been diagnosed with ALS. I don’t have a cure for you, but I do have a device that I can implant that can help you communicate.’”

That conversation is tantalizing. But there are no guarantees that these systems will work as hoped, or that they will be widely available. Such implants would probably come with hefty price tags, invasive brain surgeries and perhaps more demand than the few capable clinicians and technologists could meet.

And there are underlying questions about the brain itself. One concern is that changes to the brain over time might render neural signals unreliable, especially as diseases progress. If that happens, a person’s interface might produce a scrambled message — or be left without enough neural data to decode.

“These are real challenges, and the field is working on them,” Stavisky says. “But they’re not solved yet.”

Despite these hurdles, the push for voice-restoring technology can’t come soon enough for people who have lost theirs. In September, researchers reported that the woman with the Paradromics implant had a message for them. “OK,” she used her brain-computer interface to tell researchers. “I have a lot to say.” With her system, she was able to call her grandchildren on the telephone.

And Harrell is now living a life that looks very different from the one he might have imagined six years ago. Although his disease keeps him close to home, he can now share his thoughts with the world — and his family — instead of locking them away inside.

“The ability to scream when you are running and want to get out some stress, the ability to say sweet nothings into your partner’s ear,” Harrell says. “These are the things that life is all about, and when you are robbed of those times, you want them back at all costs. And this is why this is good enough for now.”

But he too has a vision, a day when these devices are commonplace, “when I am one of many, instead of being one of one.”

---

For more about the human voice and to hear from Casey Harrell himself, listen to The Deep End podcast Season Two, called Talk to Me. It’s free wherever you get your podcasts.