Information Flow in Music (June 28, Wednesday)

Abstract: The theory of Uniform Information Density, originally proposed for language, is very suggestive with regard to music as well. The theory states that communication is optimal when information is presented at a moderate and uniform rate. Three predictions follow for music: (1) low-probability events should be longer in duration than high-probability events; (2) low-probability events should be mixed with high-probability events; (3) an event that is low in probability in one dimension should be high in probability in other dimensions. I present evidence supporting all three of these predictions from three diverse areas of Western music: Renaissance counterpoint, expressive piano performance, and common-practice themes.

Three Things That Make (or, How to Ruin) a Great Melody (June 29, Thursday)

Abstract: Studies have shown that melody is among the most important and valued aspects of musical experience, but little is known about what makes a great melody. In this talk, I present three features or strategies that I think many great melodies have in common, focusing on Western music of the “common-practice” period (the 18th and 19th centuries). 1) When patterns are repeated within the melody, they tend to be altered or re-contextualized in such a way that makes them less probable in some respect (for example, in their intervallic pattern or their harmonic context). 2) Major and minor modes are mixed, either by introducing elements of the parallel key or by implying other keys of the opposite mode. 3) Cadences incorporate motivic ideas from earlier in the melody. I will illustrate these three strategies using a set of well-loved melodies from the common-practice repertoire. Previous music research provides indirect support for all three of these strategies as being favored by composers and listeners. My claim that they contribute to the “greatness” of specific melodies is conjectural, but it can be tested informally by removing them from the melodies and considering the effect. 

Timbral cues in musical and linguistic pitch perception

Abstract: Listeners integrate timbral cues (spectral shape) in pitch perception; sounds with more energy in higher-frequency harmonics tend to sound higher in pitch than sounds with less energy in higher-frequency harmonics. A flatter spectral slope generally corresponds to `tenser’ sounding voices in speech, while voices with a steeper spectral slope tend to sound ‘breathier’. Similarly, in string instruments such as the violin, the spectral slope differentiates sul ponticello (flatter spectrum) and sul tasto (steeper spectrum). In this study, we investigated whether listeners integrate spectral slope cues differently when listening to speech vs violin sounds. This contrast is interesting because while the spectral slope co-varies with the fundamental frequency (F0) in speech production, they are largely independent in instrumental music-making. 88 adult listeners participated in a pitch classification experiment involving speech and violin stimuli with identical pitch contours and spectral slope manipulations. Results showed that listeners integrate spectral slope cues in pitch perception similarly in speech and violin, with individual differences driven by listeners’ musical competence. In sum, we find evidence for overlapping domains in linguistic and musical pitch processing.

On Using Inferential Statistics with Subjective Data; Or, Resisting ‘Cuius regio, eius religio’

Abstract: Despite all the talk about interdisciplinarity, today’s academy resembles the uneasy détente between Catholics and Protestants in sixteenth-century Europe, in which a state’s religion was determined by its ruler.  The sciences and the humanities operate within incommensurable epistemic frameworks, and the overarching discipline that contains an inquiry determines its methodological rules of the game.  This is true also for studies of music; while approaches vary widely, a straightforward allegiance either to the sciences or to the humanities is almost always clearly evident.  

I will sketch an alternate possibility.  First, Michael Polanyi’s concept of the personal helps us to see that as forms of as human knowing, the sciences and the humanities both culminate in (provisional) human judgements; the burden of choice never slips from our shoulders.  One main difference lies in the nature of the data used as input for the decision-making process; in the sciences these data are mainly objective, but in the humanities the data are themselves the products of human judgments.  Second, cognitive linguistics and dynamic-systems models of human cognition both indicate that the human mind is the best instrument presently available for understanding and interpreting the products of human minds; our judgments are fallible, but we cannot do without them.  Humanists have responded to this fallibility in various ways, including embracing it, fleeing from it, and various combinations of denial and minimizing.  In the final part of the talk I will share examples, including some from my own works in progress, that illustrate simple ways in which borrowing techniques from the sciences can increase the reliability of humanistic scholarship without distorting its nature.

Topological data analysis of Korean music and AI-driven composition

Abstract: Korean traditional music is often better understood and explained by comprehending the unique composition principles inherent in individual music pieces, rather than relying on unified music theories. However, analyzing certain Korean music pieces to uncover their underlying composition principles can be challenging. In such cases, geometric or topological analysis, which is uncommon in Korean music study communities, can be useful for quantifying these principles. In this talk, we demonstrate topological data analysis of Korean music as a powerful tool for revealing the composition rules within a given music dataset, utilizing the concept of persistent homology. Persistent homology captures how n-dimensional loop structures change across different scales. Initially, we transform the music data into a graph consisting of nodes and edges. By defining a metric function, we compute the persistent homology of the graph, with a particular focus on the computation of one-dimensional homology and the examination of interconnections between one-dimensional cycles present in the music pieces. This interconnectivity can be succinctly summarized in the form of an overlap matrix. We will highlight the efficiency of this topological analysis in characterizing various Korean music pieces. Based on the analysis, we will further explore how AI methodologies can adopt one-dimensional homology to reproduce similar music pieces. To illustrate, we will utilize a popular music piece from the Jung-Ak, known as Dodeuri music. Additionally, we will briefly show the extension of this approach to Western music.

Writing the Right Sounds: Lessons from Korean Language and Music for a Global Descriptive Notation

Abstract: Until the fifteenth century, the Korean language could only be written with Chinese characters, which were ill suited to Korean phonology and syntax. Then King Sejong introduced a new alphabetic writing system, hangeul, which was purpose-made to represent Korean speech sounds. The system was first promulgated in a document titled Hunmin Jeongeum, translatable as “Correct Sounds for the Instruction of the People.” Today, hangeul is admired by linguists worldwide as an exceptionally efficient script that really does capture the “right sounds.” Meanwhile, when it comes to writing down the world’s diverse forms of music, the situation is comparable to that of Korean language before the invention of hangeul. An old established writing system, staff notation, is used to represent forms of sound organization that have little in common with the Western music on which staff notation is based. As a result, the notation often shows the “wrong sounds,” and has to be corrected with cumbersome additional symbols and verbal explanations.Seeking a more efficient means of representing and comparing music from different traditions, in 2016 I took my cue from King Sejong to develop a new purpose-made descriptive notation system which I call “Global Notation.” My goal has been to provide a consistent way of specifying whatever information about sound may be wanted for the purposes of a particular piece of notation, while excluding any unwanted or irrelevant information. I will illustrate this with the traditional Korean instrumental genre sanjo, where varied melodic modes and microtonal inflections are vital to distinguishing different repertoires though not captured well in staff notation. Through this, I will invite input for the further development of Global Notation as a means of “writing the right sounds,” particularly at the corpus level.

Music perception and action through the lens of dynamical systems theory

Abstract: Music listening and music making are dynamic and interactive processes in which humans coordinate their internal and bodily activities with the sound and other perceptual information generated by others. Dynamical systems theory provides the conceptual and mathematical tools to describe the patterns of musical coordination emerging from the interaction of underlying forces and constraints such as intrinsic dynamics, intention, and learned (cultural) patterns. In this talk, I will review recent advances in the dynamical systems approach to music, focusing on the dynamical systems analysis and modeling of music performance data. As a focused example, I will discuss the studies of phasing performance in Steve Reich’s music, which requires musicians to intentionally desynchronize from each other. Systematic nonlinear trajectories of performance tempo and relative phase were observed in a case study with two expert musicians, and various phasing behaviors were observed in controlled experiments with non-expert participants. I will show that the phasing performance data can be explained in terms of the competition and cooperation between the intrinsic tendency of synchronization and the task-specific intention of desynchronization. This dynamical systems interpretation is supported by coupled oscillator models that replicate the main findings of the experiments.

The lyrics and beat alignment of Korean traditional music

Abstract: Some of the many songs of Korean music show typical examples of lyrics and beat alignment. Usually, they’re folksongs like children’s songs. These simple lyrics and beat combinations formed a simple rhythmic pattern, Jangdan. The simplest examples changed and developed continuously through various songs. Korean folksongs, Gagok and Pansori are different song genres of Korean traditional music. I will talk about how lyrics and beat alignment have changed in these genres in relation to Jangdan. They have fundamental forms and variations. In some genres, lyrics and beat alignment have to do with language. It gives us a starting point for a discussion about corpus building.

Infants in a bi-cultural musical environment show prolonged perceptual openness to music 

Abstract: Korean infants have access to both Korean and Western music in their environment as captured on a survey of infants’ daily exposure to music at home. We tested 49 12- to 30-month-old Korean infants on their preference for Korean or Western traditional songs played by haegeum and cello. Our results show that infants with less daily exposure to any kind of music at home listened longer to all music types. The infants’ overall listening time did not differ between Korean and Western music and instruments. Rather, those with high exposure to Western music listened longer to Korean music played with haegeum. Moreover, older toddlers (aged 24–30 months) maintained a longer interest in songs of an origin with which they are less familiar, indicating an emerging orientation towards novelty. Early orientation of Korean infants toward the novel experience of music listening is likely driven by perceptual curiosity, which drives exploratory behavior that diminishes with continued exposure. On the other hand, older infants’ orientation towards novel stimuli is led by epistemic curiosity, which motivates an infant to acquire new knowledge. Korean infants’ lack of differential listening likely reflects their protracted period of enculturation to ambient music due to complex input. Further, older infants’ novelty-orientation is consistent with findings in bilingual infants’ orientation towards novel information. Additional analysis showed a long-term effect of music exposure on infants’ vocabulary development.

Rhythm and the brain

Abstract: The temporal structure of sound, shaped by regularities, guides listeners to process auditory information efficiently. This structure can be observed in speech through regular accents and in music through regular rhythmic patterns. In this talk, I'll present neurophysiological and behavioral experiments on rhythm and explain how the temporal properties of sound and individual differences alter sound processing in the brain. In particular, I'll present frequency-following response studies that show how the structure of rhythm and meter selectively enhances sound at the brainstem level. I'll also talk about a cross-cultural study and discuss how rhythm is categorized differently depending on individual experience and cultural differences. Finally, through studies of the relationship between rhythm ability and Korean language proficiency, I'll talk about the mechanisms of rhythm that are common to music and language, as well as the potential of music programs for reading comprehension.

AI voice that can talk and sing

Abstract: The recent remarkable advancement of Generative Artificial Intelligence (Generative AI) enables anyone to generate high-quality content such as text, images, videos, and voices without requiring specialized knowledge or separate training. In this lecture, we will introduce the natural principles of voice generation, which is the most crucial means of communication for all of humanity, and explore the technology of voice synthesis using artificial intelligence, along with various application areas.

AI for Classical Music Performance

Abstract: In classical music, each performer has a different interpretation of a piece of music, varying the tempo, dynamics, and nuances of each note. Recently, AI technology has been actively applied to classical music, especially piano music, to reproduce the performance by accurately capturing the timing, duration, velocity, and pedal of each note from a recorded piano performance, to render a performance like a human pianist for a given score, or even to perform together with a human. In this talk, I will present recent research on automatic music transcription, performance rendering, and interactive performance.

The tonal-metric hierarchy in Western music and its psychological reality

Abstract: In Western music, there is a connection between the deep-level structure in pitch (tonality, or musical key) and time (metre, or beat). I have referred to this connection as the tonal-metric hierarchy, because it demonstrates that the tonal hierarchy (pitch) and metric hierarchy (time) are not merely present in virtually all Western music, but they are aligned such that tonally stable pitches occur disproportionately often at metrically stable points in time. I provided evidence for the tonal-metric hierarchy with a simple note-count corpus analysis of a range of famous composers in Western music (Bach, Mozart, Beethoven, Schubert, Chopin, Brahms, Lizst, Scriabin) back in 2014. Since then I have been focusing on providing evidence that the tonal-metric hierarchy is not simply a theoretical curiosity, but has psychological reality. Through a range of experiments I manipulated the alignment of tonal and metric information in melodies, and examined the effects on subjective and objective measures. Regardless of formal musical training, perturbing the tonal-metric hierarchy reduces listener ratings of goodness and beat clarity, at least when using composed melodies (as opposed to computer-generated sequences). Interestingly, the ability to tap along with the melodies shows a more complex story, because only online participants showed effects of the tonal-metric hierarchy. In this talk I will provide an overview of the tonal-metric hierarchy, including my corpus analysis and experimental work. Future promising extensions focus on more continuous variations of adherence to the tonal-metric hierarchy, and how this may influence musical tension ratings and recognition memory performance. 

The Global Jukebox: Using data to understand musical history and diversity

Abstract: The Global Jukebox is a resource that allows us to explore and quantify the diversity in human musical performance (www.theglobaljukebox.org). The Global Jukebox contains 5,776 traditional songs from 1,026 societies and is the most diverse collection of traditional music in the world. The Global Jukebox was recently digitised and re-released alongside several datasets that quantify the sound and context of songs. The most notable is Cantometrics, which codes each song for 37 variables. What can this data tell us about the history and diversity of music? First, I show how music can tell us something new about the cultural history of human groups. An analysis of 121 societies shows that musical data is structured in a tree-like way of comparable strength to language. The pattern of similarity in the musical data is unlike the patterns seen in genes or language. The absence of a relationship between these processes, alongside a pattern of tree-likeness suggests that music records alternative pathways of human cultural exchange and history.  Secondly, I use Cantometrics to identify music that is unusual. Unusual music is a song or society that produces music that is unexpectedly different from surrounding societies. I discuss the most unusual songs, and the most unusual societies. The most unusual song is a pan-pipe performance from Kurks, Russia, which leads to a discussion of motor constraints on musical diversity (due to its similarity to other, distant, panpipe songs). The two most unusual societies are closely related Berber societies. We discuss the difference in social organisation between these societies that have similar musical performances and question the influence of the social organisation on musical performance. The Global Jukebox has great potential to espouse the importance of diversity in the study of music. Here I have offered two examples, but there are many more possibilities. 

Auditory Processing: From Molecule to Culture

Abstract: Most of us live in an auditory world.  We use spoken language to communicate, we tune in to environmental sounds, and we listen to music.  Formal and informal experiences with these acoustically and functionally complex sounds are linked to our neural systems (from neurotransmitters to network) on the one hand, and our cultural traditions on the other.  In this talk, I will report a series of experiments from my research group that investigate factors influencing auditory processing and learning, including genetic profiles, neuroanatomy, neurophysiology, instructional paradigms, and culture.  I will talk about how individual variations in genes and brain may be tied to spoken language learning success, how different learners may require distinct training methods, and how listeners develop increased sensitivity to music of their own culture.  This collection of findings forms a starting point for a more comprehensive understanding of the human auditory system that takes into account the connection between the context in which we live and the building blocks of our physiology.