BendableSound

Worldwide, one in 59 children has autism, and 80% of them have motor problems, such as delayed or anticipated reactions and excessive or insufficient use of strength when they conduct a motor movement. Neurological Music Therapy (Nero-MT) uses live music to improve the timing and strength regulation of children with autism. However, conducting Neuro-MT sessions for children with autism is not an easy task given their motor and attention problems. These limitations could affect the interaction of children with autism with traditional musical instruments.

Elastic displays are interactive surfaces with touch capabilities allowing temporary deformations. Elastic displays offer a natural and casual way to interact and provide a multisensory experience that can help children with autism interact easily with music and stay focused during therapy. To the best of our knowledge, there are no studies investigating if elastic displays could help children with autism with their strength and timing regulation during Neuro-MT.

During my P.h.D. project, We design, development, and evaluation BendableSound, a large-scale elastic display enabling children with autism to play sounds when practicing coordination movements when tapping, pushing or pinching a fabric. BendableSound includes musical notes arranged in an ascendant way, and a 3D animated neon-ish background of space nebulas with translucent space-based elements, such as rockets and planets. BendableSound includes open-ended and structured activities with challenges related to rhythm and strength. BendableSound uses a 1.5 m³ PVC structure hanging a spandex fabric. Behind the fabric (inside the frame) we use a Kinect to detect users’ interactions, a short-throw projector to display the animations and speakers to play the sounds –this hardware is connected to a PC running the gameplay dynamics.

We conducted a formative pilot-study with 24 children with severe autism showing BendableSound can support Neuro-MT and may potentially have therapeutic benefits regarding attention and motor development.

Then, for around two months, we conducted a randomized controlled trial where 22 children with autism attended to 8 sessions of Neuro-MT by either using BendableSound (experimental group) or tambourines (control group). We conducted three pre- and post- motor screening test to evaluate strength control, timing synchronization and coordination. Our results indicate that BendableSound is more engaging and is more efficient in improving motor coordination, strength regulation and timing control than the use of tambourines.

These results suggest that elastic displays could be used as a Neuro-MT intervention with therapeutic benefits for children with autism. This work offers both clinical and technical implications for developing elastic displays, and reveals future challenges that need to be addressed to use elastic displays as a therapy in long-term.