Concept and Overview
KIT is a Zurich based collective made up of varied constellations of performers, poets, programmers, improvisers, architects, soundtrackers and makers; we specialize in the production of musical and/or interactive theater, music, and multimedia pieces (typically a combination of all of the above).
My role in the group is as the resident 'nonsense engineer', meaning I focus on the technical realization of absurd devices for the sake of art. This typically entails hacking everyday objects with sensors, microcontrollers, and sometimes Raspberry Pi single board computers, and is how I developed (or perhaps formalized) my research interest in Augmented Objects.
→ jump to:
Augmented Objects
I define an Augmented Object as an everyday object whose functionality has been modified or extended beyond its normal scope, allowing it to interact with its environment through the addition of sensing and/or computing capabilities in order to produce an output that is artistic or creative in nature. This concept is closely related with, and could perhaps be considered a subset of Turchet et al.’s notion of (the Internet of) Musical Things. I have created several such objects for our performances, including:
Audio-reactive Light (2020) - A table-top light fixture that is flashes when an envelope follower exceeds a threshold (a collaboration with Dalius Singer)
Biking Through Sound (2020) - A prepared stationary bicycle that enables the rider to travel through a virtual spatial audio scene rendered on 4 speakers positioned in front, and behind them as well as on their sides.
Talking Punching Bag (2020) - A prepared punching bag that triggers the playback of sampled sounds when one of three zones was struck
Motorized Lights (2019) - Light fixtures augmented with remote controllable hanging kitchen devices such as juicers, immersion blenders, and coffee grinders.
Speaker Tables (2019) - Hanging tables made from 25 mm thick honeycomb cardboard panels outfitted with exciters that enabled the entire structure to function as a large speaker
Sonic Vacuum Cleaner (2018) - A ‘playable’ vacuum cleaner with an integrated two channel sample player/synthesizer (with one speaker mounted in the body of the vacuum and the other worn in the performer’s costume), controlled via an interface built into the handle
Technical Implementation
Biking Through Sound
Overview:
I created my Biking Through Sound installation for Medusen (2020), a libretto homemade in the manner of a search engine around the word ‘meduse’, implemented as a multi-perspective performance, which includes a video-walk, audio-walk, treasure-hunt, a guided city tour, a concert, and several installations. It featured a bicycle mounted on a trainer stand with a dynamo power generator connected to an arduino to sense the speed at which the rider was pedaling. In order to accomplish this, I first preconditioned the signal using a half-wave rectifier circuit, then applied a smoothing filter in the Arduino sketch (in a future version I plan to compute the frequency of pulses once the maximum amplitude is reached to enable the measurement of a wider range of speeds). The resulting speed data was then sent via serial over usb to a Raspberry Pi where Pd executed the following two sound processes:
The speed of bike controlled the playback rate (directly) and amplitude (inversely) of a Mozart sample, so pedaling at full speed would play the sample at its normal rate at a reduced volume and slowing down would lead to a decrease in pitch and increase the volume, similar to a turntable power cutting effect.
Four virtual sound objects (various sources of dialog related to the show) were placed around a loop that the cyclist would ride through as if they were travelling around a track with the various sound sources located on either the left or right side of their path. The rider's position was stored as a looping counter which ranged from 0 to 1023, and the speed at which they rode determined how quickly the counter incremented; this implementation allowed the rider to stop and listen to the individual sound sources if so desired. Finally, the virtual sounds were positioned at various locations along the path (at values between 0 - 1023), and the distance between the rider's position and the sound was used to scale the source amplitude, and in turn, create the panning effect. This process was carried out for each of the four speakers individually; specifically, this required setting the left and right speaker locations equal to the current position, while the front and back speakers were assigned a location of +/- 25 from the current position.
The figure below shows an overview of the second process. In this example, the rider (at position x) will hear sound 1 (at position s1) from the front speaker, as it has entered the range of the sound's amplitude envelope (computed as d_front). As the rider continues to move forward, the sound will subsequently be played from the left speaker and rear speaker, before fading out from all three in the same order. Sound 2 then follows a similar process, only it appears on the rider's right side.
Software and Hardware specs:
Main computer: Raspberry Pi 4 Model B - 4GB
OS: Patchbox OS from Blokas Labs
Incredibly easy to set up, including the use of external audio interfaces
Allows for headless operation (starts default patch without monitor or keyboard on boot-up)
Software: Pd Vanilla
Audio interface: generic turquoise blue "5.1 Channel USB audio interface" available on eBay, aliexpress, etc. for $10-15
(NOTE: an external interface is recommended when working with audio on a Raspberry Pi, as the sound quality of the internal minijack connection on the Pi is not great...)
Arduino Nano used for control inputs and visual feedback
Speakers: 4x JBL clip 2 bluetooth speakers (connected via minijack input)
Bike + dynamo mounted on a trainer stand
Sonic Vacuum Cleaner
Overview:
The first such object I created was a “Sonic vacuum cleaner” I built for Mariana Vieira Grünig’s 2018 Roman Clemens Prize winning MA Theater Project Des Astres: Where is Melissa Gordon? The vacuum cleaner featured a Raspberry Pi single board computer running a Pure Data patch that controlled two independent audio channels - one for the speaker mounted in the body of the vacuum itself, and one for the speaker hidden in the performer's costume. Channel 1 simply played white noise (as an interpretation of the sound of a vacuum cleaner), while channel 2 featured a sample playing and synthesis engine that either played one of eight roughly minute long mini-compositions composed using Ajax Sound Studio’s Cecilia 5 software and Adobe Audition, a sine wave, or the sum of two sine waves, one with a fixed frequency and the other controlled by a LFO, thus creating an accelerating and decelerating beating effect. Audio controls and visual feedback were integrated into the handle of the vacuum cleaner. Two faders and three push buttons were used to control the output gain of each channel and the sound source of channel 2, respectively. When switching between samples, a 1 second in/3 second out crossfade was applied to the sources for a smoother transition.
Software and Hardware specs:
Main computer: Raspberry Pi 3 Model B
OS: Satellite CCRMA Raspbian build
Easy(ish) setup/integration of system and connection with Arduino
Allows for headless operation (starts default patch without monitor or keyboard on boot-up)
Software: Pd Extended
Arduino Nano used for control inputs and visual feedback
Audio amplifier: Sure TPA3118 2 x 30 Watt Class D
Speakers: 2x Visaton FR 10 HM 4Ω
Battery: ZIPPY Flightmax LiPo 8000mAh 4S1P 30C
An updated version of the vacuum cleaner replaced the battery with an AC power adapter
System diagram
Photos
