Timings: An Internet Dance

The performance occurred partly within the Unity 3D computer game platform. New motion capture recording and playback software for the Microsoft Kinect camera was developed by UI doctoral student Tony Reimer that allowed for real-time control of 3D characters within the Unity environment, as well as playback of live-recorded gesture loops. The loops played at the same time as the live dancing, giving the audience an experience of different time levels at the same moment.

Timings: An Internet Dance was quite a long time in the planning stage before any work was done on executing the plan. Over a span of several months, John Toenjes held online discussions with collaborators Yukihiko Yoshida and Keiji Mitsubuchi about ideas pertaining to various cultural, identity, and Internet- and computer-specific issues. Eventually the idea developed of a performance with the effects of timing on these issues at its core, using the latency inherent in transmitting various media types as a way to metaphorize this concept. 

Looping became a central structural idea for the dance. To facilitate this and to make possible the transmission of different media types for the same dance gestures, a custom Macintosh application was developed by IJPAN Graduate Assistant Tony Reimer and Technical Director John Toenjes, which eventually came to be known as SKTTR (SKeleTon TRacker). This application worked in conjunction with a Microsoft XBox Kinect camera to track dancers’ movements in a prescribed area. These movements and the dancers’ positions within the field of view were immediately transmitted to another Macintosh application called Unity, as well as to the local and the remote musicians, and the Unity applications that were running in the remote locations. Within Unity were 3D avatars of the dancers that responded in real time to the incoming OSC data. Thus the large 3D avatars within Unity moved in the same way that the dancers moved, either as they were dancing, or in a loop, recorded and played back by SKTTR. 

Real-time video streaming of the dancers among all three locations was also employed so that the dancers could respond to their counterparts in the other locations. The dancers responded either to the real-time video stream, or to the real-time avatar dancing, or to the avatar loop recording. Thus there were a multiplicity of options for eliciting interactive response from dancers at remote sites.

Professor Toenjes traveled to Tokyo to test the SKTTR app with the Japanese dancers and technical collaborators. He brought with him a Macintosh computer and iSight and Kinect cameras to supply the hardware and software necessary for this performance to occur in Japan. Our Japanese collaborators had only Windows OS computers, so it was necessary to transport the computer and cameras directly to them in order to train them in their use for the performance to take place. 

This performance also used multiple screens and multiple projections for extra depth, an approach that was first implemented in the IJPAN production of Sayonara and I, Worker. This proved again to be a successful way to ‘augment’ the reality of online performance. 

Timings, An Internet Dance was performed on Thursday Dec. 12, 2013, at the Krannert Center for the Performing Arts. Sponsored by IJPAN, the Illinois Japan Performing Arts Network, the Harry D. Castle Fund, Dance at Illinois, and eDREAM.