Visualizing Vocal Emotions

@KTH Royal Institute of Technology, Master thesis project

Supervisor

Prof. Roberto Bresin, Sound and Music Computing Group, Division of Media Technology and Interaction Design

Examiner

Prof. Sten Ternström, TMH Division of Speech, Music and Hearing

(Full version master thesis available at KTH DiVA)

This project investigates the audio-visual integration in singing. The thesis introduced a software system for real-time visualization of singing, aimed at enhancing the musical expression of emotions and fostering engagement.

Musical expression refers to the expressive form embedded in performance, achieved by the manipulation of a set of musical characteristics including the pitch, loudness, attack, etc., named acoustic cues. This project is focused on applying computational methods to quantify and visualize the musical expression in singing voices. The visualization was built on a parametric mapping between acoustic cues and graphical parameters. A user study was conducted to evaluate the system.

My Role

Thesis author.

Independently developed the software system using Unity and Pure Data; Designed and implemented the parametric mapping.

Designed and conducted user study with 19 participants, involving user test, questionnaire, interview, and qualitative and quantitative analysis. 

Dates

Apr. 2022 - May. 2023

Real-time audio feature extraction in Pure Data