INTERACTIVE SONIFICATION OF MOVEMENT TRAJECTORIES DURING MOTOR REHABILITATION THERAPIES
INTERACTIVE SONIFICATION OF MOVEMENT TRAJECTORIES DURING MOTOR REHABILITATION THERAPIES
Embedded Files
Problem: Interactive sonification is an effective tool used to guide individuals when practicing movements. Little research has explore the affordances of continous and discrete sound structures in supporting motor therapeutic interventions for children with autism who exhibit motor impairments. The goal of this research is to study if children with autism understand the use of such sounds during motor therapeutic interventions, its potential impact of interactive sonification in the development of motor skills in children with autism, and the feasibility of sonifying an elastic display and a mobile phone to be used in specialized schools for children with autism.
Hypothesis: We hypothesize continuous sounds will provide better guidance for the trajectory of the moment while continuous sounds will encourage repetitions independently of the technology being used including an elastic display and a mobile phone.
Keywords: Elastic display, depth camera, accelerometer, mobile sensing, interactive sonification, user studies, movement, ASD, children, motor coordination and rehabilitation
Sound structures ..
Push movement
Push movement
Initial position: The child brings his/her arms to her chest
Trajectory: The child stretches an arm forward in a horizontal way (i.e., parallel to the floor)
End position: The child has an arm fully extended forward in a horizontal position (i.e., parallel to the floor)
Lateral movement
Lateral movement
Initial position: The child is standing with his/her arms down
Trajectory: The child moves each arm from down to up “swiping” in a lateral movement and drawing a half-circle in a trajectory
End position: The child is standing with the arm raised on the same side where the movement began
Cross-lateral movement
Cross-lateral movement
Initial position: The child is standing with his/her arms down
Trajectory: The child moves each arm from up to down up crossing the right arm from right to left and the left arm from left to right
End position: The child is standing up raising his/her arm on the opposite side where the movement began
A child with autism using the three sounds developed to be used with the Bendable Sound Prototype
Push movement using the mobile phone with the three sounds
Lateral and Cross lateral movement using the mobile phone with the three sounds
Push movement using BendableSond with the continuous sounds
Lateral and Cross lateral movement using BendableSound with the discrete sounds
Evaluation methods: We conducted two deployment studies in Mexico using the mobile sensing and the Bendable sound with the designed sounds. In the first study, six children with autism were asked to perform the forward reach and lateral upper-limb exercises while listening to three different sound structures (i.e., one discrete and two continuous sounds). In the next study, nine children with autism were asked to perform upper-limb lateral, cross-lateral, and push movements while listening to five different sound structures (i.e., three discrete and two continues) designed to sonify the movements.
A child with autism using the designed sounds in the mobile phone
Outcomes and results: Results showed that children with autism exhibit awareness about the sonification of their movements and engage with the sonification. In particular, the use of discrete sound structures engage the children in the performance more repetitions while continous sound structures increase their ability to perform the movements correctly. We found out BendableSound better provides support to perform the push movement while the mobile phones outperform in supporting the practicing of lateral movements.
We thank other collaborators involved in this project including Ana Tajadura, Nadia Berthouze and Anessha Sing from UCLIC, UCL and our sponsors:
Page updated
Report abuse