Software Development

Our research on audio-visual integration is greatly accelerated by the use of customized testing software.  This software offers the ability to generate novel point light displays mimicking the striking motions used by marimbists.  This not only captures the illusions experience when seeing an actual musician (Schutz & Kubovy, 2009a), but it allows for novel manipulations such as creating hybrid gestures involving components of the short and long striking motions (Armontrout, Schutz, & Kubovy, 2009).  Using this software, we are able to explore audio-visual integration using ecologically-relevant biological motions.  However, we are also able to precisely control all aspects of these synthesized motions (motion path, acceleration/velocity, duration of movement, etc), facilitating a rigorous deconstruction of these gestures. Given the technical complexity of implementing software of this nature, the core of our system was written by Mustard Seed  (a company specializing in customized solutions for complex research problems in industry and academia).  We then call upon this powerful engine/core to create many individual experiments exploring different aspects of audio-visual integration.  This approach has led to numerous conference presentations and published papers.  The architecture of this system is diagrammed below (left) along with a video demonstrating the degree to which the software accurately captures/mimics a marimbist's striking motions (right):