User-Centered Design for Inclusive Multisensory Experiences

Music Tech, User Research, Accessibility.  

Publication (journal paper), data and files, publication (conference paper), Sound Forest project page. 

Python & R (data processing and analysis), SuperCollider (sound synthesis), Tombstone (accelerometer measurements). 

Participatory design is a crucial component in developing Accessible Digital Musical Instruments (see my previous work on ADMIs). However, limited research exists on adapting these methods for pre-verbal children with Profound and Multiple Learning Disabilities (PMLD). In this project, we applied data-driven participatory design methods to tailor and evaluate the Sound Forest installation at the Museum of Performing Arts (one of Sweden's most visited museums, attracting 265,421 visitors in 2023) together with a group of children with PMLD. 

This work aimed to develop a customized multisensory musical experience tailored to a group of children at the Dibber Rullen School, focusing on effective methods for facilitating communication about music and haptic feedback. A central focus was to use behavioral data, observational metrics, and user feedback to refine the existing installation. 

This work was a long-term collaboration with Dibber Rullen School  and the Museum of Performing Arts. The project employed a mixed-methods design, combining participatory design with proxies (PDwP) and qualitative interviews to capture in-depth insights into user needs. The initial phase involved physical characterization of the installation's vibrating platforms using accelerometers, complemented by observational data of the children's interactions during music lessons. To gain a holistic view of engagement and preferences, we also conducted teacher interviews, distributed questionnaires to parents, organized music listening sessions, and used body maps to facilitate discussions about musical vibrations felt in the body. 

For the final evaluation, we conducted an experiment comparing different sensory conditions, collecting video recordings, real-time sensor data, and structured post-experiment discussions.

Our multisensory design successfully provided a relaxing and yet dynamically evolving sonic experience. Observational data highlighted varied interaction strategies among children, emphasizing the importance of designing for individual preferences and sensory needs. While aiming to promote inclusivity, the installation still presented some accessibility limitations, offering valuable directions for future improvements. This study underscores the value of a multifaceted, data-informed approach in developing multisensory experiences tailored for pre-verbal children with PMLD, demonstrating how designs can be refined to foster more accessible and engaging interactions.

After a successful experimental phase, the Museum of Performing Arts integrated our multisensory design into its permanent Sound Forest exhibit. Since the installation's inception in 2016, I have played a pivotal role in evolving its haptic component, ensuring its continued accessibility for all visitors.