Publications

When Taylor Swift moved to Nashville, she adopted some features of the Southern dialect – like a shortened pronunciation of the “i” vowel (“ride” would sound more like “rod”), and more fronting of the “oo” vowel (“new” would sound like “knee-you”). This might reflect her integration into the Nashville region and also her connection with country music, where Southern accent is a core feature. She lost some of these features when returning to Philadelphia. After moving to New York (and at the time she began speaking more about important social and political issues), Swift spoke with a lower voice pitch.

For a person who uses a cochlear implant, the effort that it takes to mentally fill in a word that was missed can be alleviated when the talker’s face is visible. Specifically, the visible face cues shorten the time that a person spends lingering on the effort to fill in the word. This is important because it helps us understand the reason why facial cues are beneficial – it’s not just seeing the shape of the lips to determine specific sounds – it’s also the speedy recovery from entire chunks that were misperceived.

When a person with a cochlear implant has to guess at a word that they missed, you can’t tell just by listening to their voice. We found this in an experiment where we forced them to repair some missing words or gave them the full sentence, and recorded their repetition of the sentence. Their verbal responses were heard by over 170 virtual conversation partners. Nobody could reliably judge when the CI user was repairing a missing word, even though we know that this process involves exerting extra effort. This is important because it means that you might be talking to a friend or family member and not know that they are struggling to piece together fragments of words and sentences that they hear, which can be a frustrating experience for them.

Most research focuses on factors that make listening more difficult. This study shows a factor that makes listening easier. The effort of mentally repairing a missing word during a sentence can be alleviated when you know the content in advance. This is important because it reinforces the idea that giving a person some contextual clues about the topic of conversation can help them avoid getting stuck figuring out a missing piece.

When you miss a piece of a sentence and have to piece it back together in your head, it’s harder when that missing word is early in the sentence. This is important for two reasons. First, you want to get someone’s attention before you start speaking to them, because if they miss the first word or two, they might not recover to understand any of the other words. Second, it’s important for how we evaluate hearing; usually we just add up the percentage of words that a person misses, but this study suggests that some of those words are worth more than others, so it shouldn’t be a simple tally.

There are moments when you have to fill in a missing piece of a sentence when you miss a word, and this is especially common for someone who has hearing loss. This study shows that, for a person who uses a cochlear implant, the effort of repairing part of a sentence involves effort that carries over to cause mistakes on the next thing you would hear. This is important because we evaluate a person’s hearing ability using singular sentences, and a person can appear to understand that speech perfectly well because they mentally reassembled it – but this strategy would not work during a regular flowing conversation, because the speaker already is onto the next sentence and you are still catching up to the last one.

Listening is a process that unfolds moment-by-moment. So if we are interested in measuring the effort of listening, we can understand it more deeply by considering how our measurements track momentary changes that reflect the key information a person is listening to.

Sometimes experiments use methods that have been firmly established by many repetitions of previous work. This can be a sign of strength, but it can also be a major weakness if some of those previous studies used a flawed method, or if their method was misunderstood. This paper reviews the ways in which some very common research practices in speech communication have strayed from their original value.

When an audiologist tests a person’s ability to understand speech, the standard way to measure it is in the percentage of words repeated correctly. This study shows that a person can correctly repeat a word even in the case that they did not originally hear it. That process looks successful from the outside, but we show that it demands extra effort.

One of the ways that we identify words is hearing the pattern of stress (like the difference between SUBject and subJECT). Stress is conveyed by higher pitch, longer duration, and louder vowels, in addition to slightly different pronunciation of the vowel. This study shows that people with cochlear implants are not using pitch as much as people who have normal hearing, and instead place increased emphasis on duration and intensity cues.

Vocal fry (i.e. creaky voice) is one of the many ways in which people mark information in their speech. This information includes marking the end pf phrases, indicating the importance of specific words, conveying the meaning of a word in a tonal language, and also conveying social information. This paper calls out the common practice of criticizing vocal fry as a way to undermine a person’s credibility or professionalism. We address the false notion that vocal fry is unsafe, and call attention to the idea that criticism of vocal fry is often a veiled criticism of groups of people who use vocal fry, even though everyone uses it.

People who have hearing loss typically exert extra effort to understand speech, and this is a problem that has gained widespread recognition in scientific and clinical fields. However, there has been a tendency to think that effort is just a reflection of how accurately a person hears words (i.e. more mistakes = more effort). This study shows that there are situations in which effort is high despite no mistakes, and where effort is low despite more mistakes. We provide a better framework for understanding why listening can be effortful, which is based on how a listener constructs sensible meaning out of what they hear. Sensible perceptions can be effortless even if they are wrong, and perception that is only off by one sound can cause mental chaos if it results in a baffling combination of words that don’t belong together.

A person who uses a cochlear implant can recover more quickly from the effort of understanding a sentence when the sentence has some contextual cues (if you hear “sweep the floor with a broom”, the words all work together). This study shows that when you speak more slowly, the benefit of context is greater. This is important for two reasons. First, it reinforces the notion that people with hearing loss benefit from speaking slowly and clearly (not from shouting). Second, it helps us understand *why* there is a benefit – it’s not just that individual words are more clear – it’s that the words are used to better cohere the sentence into a sensible unit where all the words tie together nicely.

Women and men pronounce the same words slightly differently, based on a variety of physical and sociocultural differences. We know that listeners can adapt to these vocal differences by interpreting the same sound differently if it were produced by a woman or a man (sort of like interpreting 50 degrees as warm if you live in Minnesota, but as cold if you live in San Diego). This study shows that we make these interpretations not based on the pitch of the talker’s voice, but rather on the basis of how large we think their vocal tract is.

As you hear a sentence, you can predict upcoming words based on the preceding context (when you hear “sweep the floor with a …” you would predict “broom” before it is even spoken). This context-based prediction happens rapidly for people with normal hearing, and reduces the effort it takes for them to understand a sentence (compared to sentences with no meaningful context). However, people with cochlear implants do not show rapid benefit. Instead, they show benefit of context *after* the sentence has concluded, suggesting they might be waiting to reflect on the sentence on a global level and taking an extra moment to process.

One of the most challenging problems with cochlear implants is the lack of clarity in conveying frequency differences. This study shows that when this specific kind of audio distortion gets progressively worse, it causes progressively more effort to be exerted as a person listens to sentences.