One of our primary research interests is in how we enjoy
the illusion of unified experiences even when presented with
conflicting sensory information from different modalities. Given its
ubiquity and antiquity, music is a particularly interesting domain through which to explore such
questions. For example, we have discovered that expert percussionists use gestures to
control an audience's perceptual experience of their performances.
For more information on how this research applies to music performance and music education, please visit www.michaelschutz.net.
To see Michael Schutz demonstrating the illusion on television, visit the Press Room. To download sample videos to use in presentations/ demonstrations, click here. Please cite Schutz and Lipscomb (2007) as the source, and email us to let us know this was useful for educational purposes!
|To demonstrate this, we recorded a world-renowned marimbist performing
notes using both long (left video) and short (right video) physical gestures. We then crossed (swapped) the
audio and visual components of these gestures, and asked participants
to judge the duration of the sound independent of the image. In a
separate condition, we also asked them to judge note duration when
presented with the audio alone. The results indicate
that although physical gesture length does not affect note acoustics, it does affect note perception. In other words, expert
musicians use this illusion to control an audience's musical experience
beyond that which is acoustically possible!
Duration ratings (y-axis) of
marimba notes presented as audio alone (left) and audio-visual stimuli
(right). Notes produced by long gestures (red) are not rated any
differently than notes produced by short gestures (blue) when presented
as audio-alone. However, they are rated differently when presented with
the accompanying gesture.