Research Themes

In Uncertainty and Surprise Jointly Predict Musical Pleasure and Amygdala, Hippocampus, and Auditory Cortex Activity , we used an unsupervised statistical learning model to analyse the expectancy of over 80,000 pop song chords and showed that musical pleasure is modulated by the interaction between how much we can predict an upcoming chord, and the extent our expectations are violated. This is in line with Predictive Coding, and we further show that expectancy-driven musical pleasure is related to activity in brain regions processing emotion and auditory information. 

Although our previous work demonstrated the role of expectancy in shaping our appreciation for music, how listeners formed musical expectations remained unclear. We addressed this in Distinct roles of cognitive and sensory information in musical expectancy, where we compared various computational models of musical expectancy using Bayesian methods. We found that listeners formed musical expectations based on both short-term sensory mechanisms and long-term representations of music structure, although with a larger contribution from the latter.

In Decoding musical pitch from human brain activity with automatic voxel-wise whole-brain fMRI feature selection, we proposed a new two-stage thresholding method that automatically selects maximally relevant voxels in the brain for decoding tasks. Our method is interpretable on the voxel-level, and capitalises on local functional organisation whilst enabling feature selection across the whole-brain. We applied our method on an fMRI dataset where listeners heard single instrument notes, and demonstrated a two-fold improvement in pitch-height decoding when using features selected with our method compared to the traditional regions of interest approach.

One advantage of studying brain function with multivariate patterns of neural activation compared to typical mass-univariate approaches is enhanced sensitivity. In Neocortical substrates of feelings evoked with music in the ACC, insula, and somatosensory cortex, we showed that neural activation in the cingulate cortex, anterior insula, and somatosensory cortex also encode information about emotions evoked by natural musical stimuli.

We provided the groundwork towards biosignal-inspired music recommendation in Rapidly Predicting Music Artistic Expression Preference From Heart Rate and Respiration Rate. By interpreting predictions from a gradient-boosting model, we inferred non-linear relationships between music expression preference and respiration and heart rate.

In Semi-supervised Violin Fingering Generation Using Variational Autoencoders, we proposed a variational encoder model that learns to generate violin fingering information from both labelled and unlabelled data. The key insight is to treat missing fingerings as an additional latent variable following a Gumbel Softmax distribution that could be used by the decoder for reconstructing pitch and timing information. We showed that our model successfully replicated the style of a human performer, and achieved state-of-the-art performance with only half the amount of labelled data.

In An Interactive Automatic Violin Fingering Recommendation Interface, we provided a simple Python-based GUI that enables users without programming skills to upload and annotate violin sheet music saved in MusicXML. The interface allows both customised and automatically generated fingerings to cater for the unique needs of each musician. Scores with annotated fingerings can then be exported into PDF files.