Leonardo's Chimes

Performance: Version 1, excerpts

CREDITS

Artistic Conception John Toenjes, David Marchant

Dancer David Marchant

Musician John Toenjes

Vee-Jay (version 2) Ben Smith

Programming Ben Smith, Rick Valentin, John Toenjes

Performances: (version 1)

Oct. 2006 Washington University, St. Louis
Nov. 2006 UIUC Krannert Center for the Performing Arts
March 2008 SPARK Festival of Electronic Music and Media, Minneapolis

Performances: (version 2)

Featured Performers at the Ingenuity Festival, Cleveland, OH, July 2008

Documentation of Performance: Version 2 Part 1

Documentation of Performance: Version 2 Part 2

NOTES

Leonardo's Chimes is an interactive dance, music, and video piece wherein the movement creates the music and the music simultaneously informs the movement choices and vocabulary. Video effects also influence the quality of the movement. The “dancer” triggers music via hot spots on the stage, and the “musician” and computer interactive system together decide which sounds will be heard. The system makes pitch decisions based upon number of times the “dancer” intersects each hot spot. The “musician” records sequences in real time on a custom made, light sensitive interface. The “dancer” then responds according to what he hears and sees, altering his movement to carry the musical improvisation forward, spontaneously creating a layered large-motor-movement-generated composition.

The technology used in Leonardo's Chimes can be thought of in four categories: Motion Tracking, Custom Instruments, Video Display, and Interaction Programming:

Motion Tracking is done via an overhead camera whose image is fed into either Max/MSP/Jitter/SoftVNS, or Isadora® interactivity software (depending on which version of the dance). We use either presence tracking or color tracking to determine if the dancer comes into contact with the hot spots that are drawn within the video frame. In version 1, the hot spots are arranged in a horseshoe pattern around the dancer. In version 2, these hot spots have become dynamic, themselves moving in relation to the dancer and to each other, giving him many more possibilities for interacting with them.

Custom Instruments have been designed and built by John Toenjes for Leonardo's Chimes. There are two in use: the Control/Recorder which effects the music, and the VideoLyre which controls the video. These instruments were designed for theatrical effect, and to blur the distinction between dancer, musician, v-jay; and between performer and technician.

Video Display is integrated into the conception and realization of Leonardo's Chimes. It is present in the same physical space as the dancer, and the quality of the video effects influence the way the dancer moves, much as a painter reacts to his creation as it unfolds.

Interaction Programming is the mechanism for making the environment responsive to movement, for integrating the instruments into the creative mix, and for controlling the interactive computer system's realtime response to the myriad changing situations throughout the dance. The software for this is again, Max/MSP/Jitter/SoftVNS, and Isadora.

The Instruments Used in Leonardo's Chimes

The Control/Recorder

Conceived of and built by John Toenjes, the Control/Recorder allows the “musician” to work with the computer interactive system to decide which sounds will be heard during the dance work Leonardo's Chimes. The dancer triggers hotspots on the stage, tracked from an overhead camera with Isadora software. Positioning information is sent via OSC to Max, which translates it into MIDI note commands. The system makes pitch (MIDI note number) decisions based upon which hot spot is contacted, and changes the pitch according to the number of times the “dancer” intersects each hot spot.

The “musician” decides which sounds the dancer is playing at any one time (by selecting various MIDI channels). He also records musical sequences created by the dancer in real time, creating layers of musical loops over and against which the dancer “dances a music solo.” Using the Control/ Recorder, he can record sequences, turn them on and off, control the relative volume of the different sound layers, and discard sequences. This allows him to have a duet with the dancer, who must listen intently to the changing musical elements being fed to him at any time.

The instrument consists of a box made of stained birch plywood which holds most of the electronics and some magnetic switches. Upon this box rest two small elevated platforms, each of which house photocells which respond to the shadow of the player's hand. The left platform controls the selection of MIDI channels. Glowing brighter as the sounds get louder, small LED lights above each of the photocells give feedback on MIDI channel volume. The right platform selects which musical sequence is recorded and played back. LED's on this platform indicate which sequences are currently activated.

The wooden wand contains a magnetic switch that is used to select the various functions available from the interface: record/play/stop/mute all sequences, start/stop audio, MIDI bank select, and automatic/selectable MIDI channel. It also controls meta functions such as sound system on/off, lights up/down, and stop/start all sounds. LED's on the box indicate which control functions are active.

This wand now also contains an infrared LED on the end. Its vertical position is tracked by a Wii® controller to control the volume of each MIDI channel. The player selects which channel is being effected by casting a shadow with his left hand, and moves the wand up and down. When the desired volume is reached, he moves his shadow, and the volume sticks.

Inside the box are a Teleo® network and a MIDItron®, which provide digital and analog inputs and outputs and the circuitry to communicate via USB over ethernet to the interactivity programming software, usually Max/MSP.

The Control/Recorder seen from above.

John Toenjes playing the Control/Recorder (in this earlier version, the player's right hand was tracked to control MIDI volume)

The VideoLyre

VideoLyre front view

VideoLyre from player's position

Built for Version 2 of Leonardo's Chimes, the VideoLyre allows the “Vee-Jay” to mix the live video effects that are seen during the dance work Leonardo's Chimes, part of the Inventions Suite. The VideoLyre consists of strings mounted on a wooden frame that go through a base of wood to the workings inside the case.

Each string is connected inside the case to springs that keep tension, and to a length of conductive rubber stretch sensor through which a small voltage is sent from and back to a MIDItron®.

As the string is pulled up, the conductive rubber is stretched, causing a voltage drop along its length. This is translated by the MIDItron into MIDI commands that control the amount of effect on the video. Switches mounted on the case let the Vee-Jay choose which effects are activated at any one time, and to select adjustment parameters.

The intent of the VideoLyre is to expand the identity of the Vee-Jay from that of solely a mix artist, into one of onstage presence and "hand-dancer." Its form also blurs the distinction between musical instrument and technical video equipment, lending to the aesthetic of total integration of the arts.

Report abuse