A voice matters not just to the person who is speaking, but to the people listening, too. A study on moms and their daughters demonstrates the power of a voice by showing that moms’ voices reduced a measure of stress in the daughters. The same wasn’t true for texting.
In the second episode of The Deep End: Talk to Me, you’ll hear more about the ripple effects created by a lost voice. Jules Rodriguez says that as his voice slipped away, it felt like he was fading. But could a new, more realistic digital voice step in and change his story?
Hear about a tough medical decision Jules and his family were facing, and whether a sophisticated voice clone could influence his choice.
[reporter audio] Hi, this is Laura calling. This is Laura Sanders from Science News. Is now an OK time to talk about your paper coming out? I would love to start with just the very big picture question of why you did this study. Why did you do this study? What were you hoping to find?
Laura Sanders: I’m a neuroscience reporter with Science News, which means I spend a lot of time reading research papers about the brain. One of my very favorite scientific papers, and there are a lot of them, so this is really saying something, is a study published way back in 2012.
[voice] Researchers at the University of Wisconsin have found the sound of a mother’s voice triggers a physical response.
Sanders: Sixty-eight girls, all between the ages of 7 and 12, came into a lab in Madison, Wisconsin. For the experiment, these girls were asked to tell a story out loud and then perform a tricky subtraction problem in their heads. The whole time, they were being watched and evaluated by the researchers. I get sweaty just thinking about it. OK, so after that ordeal, the girls were separated into groups. Some girls got to see their moms in person and talk with them after their stressful performance. Some got to talk to their moms on the phone. Some got to text with their moms. And some just had to sit alone wondering what had just happened to them.
Later on, when researchers measured stress hormones in the girls, they found something startling. The girls who had talked to their moms, either in person or on the phone, had lower levels of the stress hormone cortisol than the girls who had just texted or had no contact.
The authors conclude that there’s something especially soothing about hearing a mom’s voice, some alchemy of familiar and calming sounds that a daughter’s brain wants to hear.
[voices] So call your mother. Good advice at any time.
Sanders: In a full circle sort of way, I tell my own mom about the study after calling her to complain about a bunch of stuff.
[phone call] The talking mattered. The texting did nothing. The phone call and the in-person lowered their cortisol levels. Well good. So, the voice. That’s what you just did for me. That’s the deal. Well I’m glad. That’s just excellent. Thanks, Mom. All right. Yeah, talk to you later. Bye. Bye.
Kate Nilson: And, you know, it’s so funny. Every time I’m with my parents, I think I need to record their voices. How can I be in this world and not have their voices recorded? So I need to get on that.
Sanders: That’s speech-language pathologist Kate Nilson, a person very familiar with the specialness of voices.
Nilson: Just for all the reasons we just talked about, right? So when I get stressed out, I can just listen to my own mom’s voice. I mean, I’m in my 40s and I still need my mom, so.
Sanders: Same.
Nilson: I need my mom to calm me down.
Sanders: I mean, yes, listen to the nice comforting sounds my mom makes on the phone when I go through my laundry list of problems.
[phone call] Yeah. I hear it in your voice. Oh, that’s sad. Get me some phone numbers and I’ll make calls. That’s pretty intensive stuff.
[intro] Hey mom, can I tell you something? The voice is absolutely an integral part of your identity. Do you want to talk? That added layer of it being you. Hello? Your voice. Hi! Your voice is you.
Sanders: This desire to capture voices that have special meaning to us, to hold them close, is a perspective I heard over and over. That’s because voices don’t just mean something to the people who make them. Voices mean just as much to the people hearing them.
Welcome to this second episode of The Deep End, Talk to Me, from Science News. I’m Laura Sanders, and today we’re going to explore what it means to lose the ability to talk.
Nilson: Your voice is you.
Sanders: Jules Rodriguez has a unique perspective on the human voice.
Jules: Your voice is different because it’s a big part of your identity. I knew early on, my voice may disappear, so we made sure to record it while I still had it.
Sanders: Jules is a podcaster, former science teacher, comedian, husband, and dad who lives in Miami. He was diagnosed with ALS in 2020. The progressive nature of the disease means that people start losing strength and the ability to move, all while their brains and cognition are unaffected. That loss includes the muscles that are required to talk.
MA: I still remember the first time. This was in 2022, the summer of ‘22, that I heard the first little crack in his voice that let me know something’s changing.
Sanders: That’s Jules’ wife, MA. When I was first connected to Jules and MA, I learned they had a podcast called The Couple Shift.
[podcast]: Well, hello there, everyone. Welcome to episode zero of our brand new podcast.
Sanders: About their marriage and themselves as they dealt with life, careers and family. I started listening at the beginning.
[podcast] My name is Maria Aleandra. I am a life and business coach, and this is my amazing hubby. Julian Tomas. And I’m a former teacher and amazing husband. He sure is.
Sanders: They were so sweet with each other. Playful, teasing, funny, yet sincere and loving. Listen to them being cute.
[podcast] So I got frustrated. And by lash shout, it’s like an outburst. It’s a [beep] what the hell? You know, that kind of deal. Do we have to bleep out? We do. Yes. Sorry. Gosh darn it. I should know better. Is that what you’re going to tell me? Yeah, that’s what my look is saying right now.
Sanders: They started recording their podcast in 2019, before Jules’ ALS diagnosis. As I listened, moving from early episodes to later ones, I got this feeling of time zooming past, just so fast I couldn’t catch it.
[podcast] We went to go see the doctor again to get his final assessment, I guess.
Sanders: Unlike the Jules and MA from early episodes, I already knew the twist, the gut punch of what was coming.
[podcast] Unfortunately, the news is that he believes it’s ALS or SMA, which is spinal muscular atrophy. It was a really hard hit in the balls. And it’s been a very difficult couple weeks.
Sanders: As I moved through the episodes, I started hearing a difference in Jules’ voice. His voice got slower, weaker, fuzzier. I asked him recently what that was like.
Jules: It’s strange with the podcast because you can hear my real voice slowly decay till we stop.
MA: Yeah. It’s really this archive of what he sounded like, little by little as it changed. And it was, yeah, like very disarming to get to that moment. And then you begin to get used to it and you sort of work around it and appreciate that it’s still there. Yeah, that was really, really difficult for me.
Sanders: Jules and MA even talk about this very process on their podcast, of having a record of their voices through time. Here’s a clip from 2024 when Jules could still talk, but with difficulty.
[podcast] To me, it’s so weird. Yes. We’re such different people and different mindsets and just different, well, lack of experience and dealing with all this. It’s so freaking beautiful, though, because you can see the whole evolution of us. It’s a living legacy of a life lived. A living legacy. A living legacy of a life lived. You and your words. Me and my words.
Sanders: In those last episodes, I was hearing the change in Jules’s voice. I was also hearing his emotions. I was hearing his weariness. Jules compares the process of losing his voice to a Back to the Future moment that any person raised in the 80s will remember. It’s when Marty sees a photograph of his family in a timeline that no longer exists.
[movie audio] Just as I thought. This proves my theory. Look at your brother. His head’s gone. It’s like he’s been erased.
Jules: For me, it felt like I was fading away. Like when Marty McFly was seeing his family fade away in the picture he had in Back to the Future.
[movie audio] Erased from existence.
Sanders: A piece of Jules was fading away. A really important piece that tells other people something about his internal state, his emotions, where he’s been, and what’s happening now.
Voices are packed with all sorts of signals. Sometimes they’re a little bit unexpected, like when a guy you don’t know all that well says “about” and you realize he is Canadian. Sometimes they’re absolutely obvious, like a strong Southern drawl. Voices can signal energy levels, whether your allergies are acting up, where you were born, and how old you are. Here’s Greg Bryant of UCLA.
Bryant: Cues are any kind of property that might predict something about the sender. You can tell, for example, how big a person is by hearing their voice. You can tell if they’re male or female generally by hearing their voice. You can tell if they’re old or young, right? So there’s all these kind of basic properties of a person that are indexed in a voice.
Sanders: Voices can carry emotions too. A smile, for example, has subtle sound cues.
Bryant: There’s a slight increase in pitch when you smile and then you change what are called formants. The formants are structures in the speech signal underlying what your perception of vowels is. They’re energy bands that are associated with resonating chambers in your throat and in your mouth. And so when you are moving your tongue around and changing your lips and doing other things going, ahowowowow, you’re changing the formants. That’s the resonating bands of energy in the voice. So when you smile, then the formants change a little bit. And it’s detectable. It’s subtle, but you can hear it for sure.
Sanders: See if you can detect it here. It’s Jules introducing an episode of their podcast in 2020.
[podcast] So my wife comes to me today, and she says, babe, I have diarrhea.
Sanders: So setting aside the content he chose to share, which was a bold choice, you can hear the grin in his voice.
[podcast] Before we get into that, though, I also want to say one quick thing. Happy birthday, my love.
Sanders: That’s because his smile pulled his lips slightly upward and changed the shape of the air coming out, thereby changing the sound of his voice. All of these signals in the voice exist because voices are made by bodies. Bodies that move, smile, change, grow older, get sick. Our voices are actually a stunningly sophisticated bit of biology.
Jessica Pietrowski: You think about how, you know, we’re just talking, we’re just voicing, we’re barely even thinking about what we’re doing, but actually upwards close to a hundred muscles are being used.
Sanders: That’s speech-language pathologist and scientist Jessica Pietrowski of UC Davis.
Pietrowski: When we go to voice, we have to take a breath. Breath comes up, the diaphragm is contracting to get that breath. Breath comes up below the vocal folds, these tissues here, vibrates the vocal folds, creates a tone. That tone comes up through the oral cavity and other spaces, and resonates. And then you have to articulate it, and, and make the sounds clear and sharp. And then you have speech. And so really, you know, we’re not thinking about this as we’re doing it, but it’s incredibly complex and intricate and really an incredible human feat.
Sanders: Right now, as I say these words to you, I’m pushing air out of my body and using my muscles to move my diaphragm, face, throat, and mouth to make tiny ripples of sound shaped with a precision far below the level of my conscious perception. This mechanical, biological, physical feat is what allows me to shape air into sounds that then move to your ears.
Sergey Stavisky: So this is a very engineering-y perspective, but one thing about voices, it is our fastest means to communicate what we’re thinking.
Sanders: Sergey Stavisky co-directs the UC Davis Neuroprosthetics Lab, focused on using technology to replace or repair parts of the damaged nervous system.
Stavisky: So it’s actually, if you step back, kind of magical. So you have some idea in your mind, and your brain is able to take that idea, form it into language. So that’s already a complicated step. And then it’s able to move almost a hundred muscles in your tongue, in your jaw, and in your larynx to vibrate air in this exquisitely precise way.
Sanders: But then, even more magic happens when those vibrations move into someone else’s ear.
Stavisky: Those vibrations are going to fly through the air and they’re going to hit someone else’s eardrum and it’s going to vibrate that, and it’s going to vibrate the hair cells. And it’s going to send a bunch of nerve signals to their brain and they’re going to first make sense of the sounds you’re making amidst all of the other sounds, whether there’s background conversation or a train rumbling by or a dog barking. But then beyond that, they’re going to not just hear the words you just said, but remember all the previous words, put them together and like get the same idea in their brain. I mean, it’s telepathy through mechanical means.
Sanders: Telepathy through mechanical means. When you step back and think about it, it’s an absolute marvel. So here I am floating these ripples out toward my microphone. And then through more technology, they are reaching you, where your own amazing machinery senses those ripples and turns them into meaning. So with these ripples, I’m putting something into your mind.
Stavisky: We have this like shared scaffolding of understanding where I can pretty much visualize anything I want and with enough words, make someone else put that into their mind. And I think that’s a really remarkable ability. And that’s like the basis of civilization and learning and science and connection.
Sanders: But there’s something happening before all of this muscle moving and air rippling, and it’s something that Stavisky just mentioned. My words first started taking shape somewhere else. They started in my brain. Certain clusters of my synapses fired as I thought about what to say. This intricate path from brain to muscles to sculpted air is behind every word we say.
Sanders: Hi Casey.
Casey Harrell: Hello there and thanks for being patient with me as I was running a bit late.
Sanders: When I met Casey Harrell on Zoom, I asked him to tell me a little bit about himself.
Casey: That is both the easiest and the hardest question to answer, right? So my name is Casey Harrell, and I am 47 years old. I grew up in the Atlanta area and have lived in the Bay Area for the past 23 years, most of them with my wife, Levana, and our seven-year-old daughter, who is going on 17. I spent most of my professional career as a climate campaigner and have went all over the globe, fighting companies who are destroying the earth and trying to bring some justice to our planet and the people who inhabit it.
Sanders: Like Jules, Casey has ALS. The disease has taken a lot from him. He can no longer move most of his body. That includes speech muscles.
Casey: I began to lose my ability to talk shortly after my diagnosis in 2020.
Sanders: With time, he could no longer talk clearly enough to be understood.
Casey: Then it became a slow-motion car crash of multiple years before almost no one would be able to understand me. A few other friends who were around often would be able to understand some of what I was saying. I relied upon my assistant and my wife to basically serve as my translators to other people. I did not go anywhere, even a video call without them, either one or both.
Sanders: Casey’s life in the Bay Area was changing in difficult, almost impossible ways. Like Jules, Casey had received an ALS diagnosis in 2020. Like Jules, he had a kid, a partner, a career, a big, full life. And then wham, life threw them one of the biggest curveballs you can imagine. These two men, separated by 3,000 miles, Jules in Miami, Casey in Oakland, were both facing big losses. And one of the biggest of all was their ability to talk.
Speech-language pathologist Kate Nilson walks me through some of the standard approaches that she uses when she’s working with people who face similar futures. There’s a process called voice banking.
Nilson: So voice banking is the idea of recording specific sentences that you might not actually say in real life, but predetermined sentences that you record that give the software the combinations of sounds that it needs to make a computer voice that sounds like you.
Sanders: That process has actually improved quite a bit in recent years.
Nilson: When I started doing this, it was 1600 sentences. Now it can be an app on your phone that you do at home. It can be a website that you go to. The software that I use most of the time is 50 sentences. It can be done in like 15 minutes. It’s really come a long way and the voices are pretty darn amazing in how how true to the person they are. But they are still a computerized voice, right? They’re they’re they’re taking combinations of sounds and splicing them together. So it sounds computerized.
Sanders: Voice banking has the benefit of being flexible. This voice can say anything a person wants, but it sounds off. There’s another approach. This one’s called message banking.
Nilson: So the idea of message banking is we just record an entire message. So instead of using that computer voice to say, “Good night, honey, I love you,” we can just record someone saying, “Good night, honey, I love you,” with their voice, like, with their true inflection and meaning.
Sanders: The beauty of these messages is that they capture the quirks of a person’s voice.
Nilson: You know, I live in Minnesota, and so there’s a lot of ooftas and doncha knows, and those kinds of things that do not come out right when you type them into a computer.
Sanders: Nilson encounters these sorts of sounds when she’s helping people save recordings of their voice for the future.
Nilson: You gotta get all those right sayings with the right, you know, “o’s” in them.
Sanders: These recorded messages are often things like an inside joke with the exact right timing or a favorite little nickname for someone special.
Nilson: It can be really emotional. And that’s, when you’re recording the sentences that are predetermined and they’re not sentences that you would normally say, that can be easy. But when your family has written down a list of meaningful words that they want to hear in your voice and you know that you’re recording it because you might lose your voice, that can be really emotional. And it can be hard. And I fully understand that, but we can also have fun with it.
Sanders: Pre-recorded jokes come up pretty often when people are choosing what messages to preserve for later use. And swear words come up surprisingly often too, Nilson tells me.
Nilson: I had someone come in just the other day, and they had an entire category of recorded cuss words. So that’s important. Well, right? I mean, you don’t want them to be monotone when you’re frustrated. You want the right emphasis to come out on that.
Sanders: It takes the sails out of it for sure if it’s monotone.
Nilson: Yeah, exactly. I mean, that’s communication, right? That’s important communication. So I guess it doesn’t really surprise me anymore, but it’s fun nonetheless.
Sanders: She says some people come to her with entire folders of swears, folders that she refused to share with me for the story, by the way.
Nilson: The best part is it was a swear folder with all these different swear words and like different sayings and whatever. And then just one of them had a call-out to a specific person. And I was like, what did that person do to you that he’s the only one that has a specific? And then he laughed and he typed out, “If you knew him, you’d understand.”
Sanders: All of these quirks, the swears, the smiles in the voice, the familiar jokes, all of these things make a particular voice special. Jules created a banked voice. He didn’t love the result.
Jules: What I learned from banking my voice was that it really wasn’t my voice. It was more like a robot trying to emulate my voice. It had no emotion or tone behind it.
Sanders: This was a problem because it got in the way of something Jules wanted to do. The turmoil of ALS forced some clarity for him about what he wanted out of life, what he wanted to spend his time doing. And one of those answers, it turns out, was comedy. After his diagnosis, while he still had his voice, he performed stand-up a few times.
MA: And that was amazing, right?
Sanders: But as his voice got harder to use, things changed. Jules could still talk, but it took a lot of work, especially since he was often using a mask for extra oxygen.
MA: He was wearing the mask pretty much the entire time. So his voice was very labored, because you have to take the breath in to utter anything. So it was very choppy. But he did have a voice.
Sanders: With time, a live performance with his original voice became impossible. The next option was to use a banked voice, but that wasn’t good enough for a comedy performance.
MA: So then he started to lose his voice and he was like, I don’t, you know, I can’t pull this off with a banked voice because it’s so robotic. It’s monotone. You can’t really do it.
Sanders: Jules couldn’t pull off a comedy show. But could there be a different way, a different voice?
I’m going to step back here and give you more context about Jules and about ALS. ALS is progressive and relentless, which means over time, more and more of the motor neurons in the brain and spinal cord are affected, and symptoms get worse.
And though this progression varies quite a bit from person to person, the median survival time after an ALS diagnosis is about 30 months. Time is short here. This diagnosis forces big choices about how to live and what a person wants their remaining time to include, for them and for their family. In the middle of everything else going on, Jules was facing a choice that many people in his situation must confront, whether or not he would get a tracheostomy.
That’s a procedure in which doctors place a tube in the throat. This tube is then connected to a ventilator that breathes for a person.
The procedure can leave a person unable to speak again, with whatever residual voice they might have left. Now, we have spent this episode on what it feels like to lose your voice. Jules was losing his. So it might seem like this part of the decision wouldn’t matter all that much. But to a person who’s losing their voice, and to their family, this moment marks an end point. It’s a hard stop.
On the next episode.
MA: He was someone who didn’t want to do a tracheostomy. He was opting for, once I’m done, that’s it. And that was our plan.
Sanders: As Jules and MA confront this decision, could a different sort of voice change the story?
Jules: So when I first heard my cloned voice, it felt like I reappeared. When my wife first heard it, she cried because of how close it sounded like me before I lost my voice.
Sanders: This is The Deep End. Talk to me. I’m Laura Sanders. If you like this podcast, tell your friends and leave us a review. It helps the show a lot.
Send us your questions and your comments at podcasts@sciencenews.org. The Deep End is a production of Science News. It’s based on original reporting by me, Laura Sanders. This episode was produced and mixed by Ella Rowen. Our project manager is Ashley Yeager. Nancy Schute is our editor in chief. Our music is by Blue Dot Sessions. The podcast is made possible in part by the Burroughs Wellcome Fund with support from PRX.
This podcast was produced with support from the Burroughs Wellcome Fund and in partnership with PRX. The podcast logo was created by Neil Webb.