Research
Research in the CAPSlab explores the psychological and neurological processing of rhythm and meter in speech production and perception. Through the lens of human language production and comprehension, we study the psychological reality of auditory imagery, and the role it plays in language perception and reading acquisition. In addition, we explore how prosodic features are produced and perceived across languages, and what cues differentiate speech from song.
Implicit prosody affects on-line reading
An enduring question in cognitive psychology is about the reality of imagery and the extent to which imagined representations are similar to perceived ones. Our work shows similarities in the processing of real and imagined sound representations. We demonstrate that readers encounter processing difficulty when their expectations about lexical stress patterns are not realized during silent reading. Eye-tracking experiments show that when participants were induced by context to expect a word with a weak-strong stress pattern, they slowed down when they read a word with a strong-weak pattern. Event-related potential studies show that neural responses to unexpected lexical stress patterns encountered during reading are similar in timing and scalp distribution to those elicited by spoken words with unexpected stress patterns during listening
Metric structure in literacy development
Literacy experts have long suggested that rhythmic, rhyming children’s books promote literacy because they emphasize the association between letters and sounds. To clarify the contribution that nursery rhymes make to early reading skill, we first need to understand what type of information readers of nursery rhymes are exposed to through their interaction with metrically-regular, rhyming texts, and how this information differs from prose. Moreover, because children routinely hear these texts before they read them, we need to understand how they are produced by readers. We explore how adult speakers cue linguistic structure in children’s poetic texts and demonstrate how adult readers use pitch, duration, and intensity variation to signal hierarchical metric structure, lexical frequency, syntactic structure, and text emphasis. We have also recently demonstrated that the same acoustic features that are used to cue metric structure in English texts are also used by Spanish speakers reading a Spanish children's book. This work shows that, across languages, through highly rhythmic, rhyming texts, pre-reading children are exposed to a rich set of linguistic cues that they can use to make strong predictions about when important information will occur, suggesting the mechanism by which these texts support reading acquisition.
Resolving ambiguity in speech and song
Listeners quickly and effortlessly categorize human sounds as speech or song, but there is considerable question about the cues that they use to do this, and how individuals differ in this process. The speech-to-song illusion is a phenomenon whereby some speech samples, with repetition, transform so that they sound like song. We've observed that listeners are more likely to hear speech segments as song when a) the onsets of the syllables create a regular rhythm, b) the relationships between pitches of the syllables are similar to those of western musical traditions, and c) the pitches of syllables are flat. We have also demonstrated that while musical experience itself does not predict the likelihood that an individual will perceive the illusion, an individual's ability to correctly identify rhythm and pitch deviations in music and speech correlates with the strength of the illusion.
Prosody predicts reading comprehension ability
Extensive work in literacy has demonstrated that children who have good prosody tend to also have good comprehension abilities. Most of this work has been carried out with children, who are still developing as readers, but very little has focused on high school students, who are already considered to be reading experts. In a large-scale investigation of high school students, we show that poor comprehenders are less likely to pause at syntactic boundaries than good comprehenders and are not as proficient at percieving prosodic cues to syntactic and semantic structure. These studies further expand our understanding of the connection between speech production and perception and reading comprehension.
Landi N, Kaswer L, Ryherd K, Van Dyke J, Krivokapic J, Breen M. Use of prosodic cues is impaired in poor comprehenders. 25th Annual Meeting of the Society for Scientific Study of Reading; 2018 July; Brighton, UK
Acoustic correlates of prosodic categories
In my doctoral work, I investigated the acoustic realization of accents and phrase boundaries, using multi-level modeling to investigate the role of duration, pitch, and intensity and the production of these linguistic elements. I explored how speakers use accents to signal different types of focused elements to listeners, demonstrating that focused words are consistently signaled with longer duration, higher pitch, and greater intensity, regardless of where in the sentence the focus occurs, but also that speakers signal different types of focus (contrastive vs. new; wide vs. narrow) with acoustic variation. In two other studies, I showed that speakers use longer duration, lower pitch, and lower intensity to signal phrase boundaries, but that the presence of these boundaries also depends on the semantic/syntactic context as well as the length of syntactic constituents that are separated by the phrase boundary. This work not only informs our understanding of speech production processes and realization of prosody through acoustics, but also informs our understanding of the semantic categories that are routinely cued by speakers and should therefore be included in any theoretical model of information status.