Performance Without Barriers and EXA: The Infinite Instrument

The goal of Performance Without Barriers is to make instruments and music making more accessible. One of their projects for making more accessible instruments was by creating a custom instrument in VR with the help of EXA: The Infinite Instrument.

Specifically, their project focused on one woman with cerebral palsey: Mary Louise McCord. By customizing The Infinite Instrument, they were able to create an instrument in VR allowed Mary Louise to make music despite her lack of fine motor control.

EXA: The Infinite Instrument

From the Steam store page:

"EXA is an immersive virtual-reality environment for composing your own music, recording loops of your performed notes and motions, designing custom instruments, performing for audiences, building up virtual bands, and more."

EXA allows any user to create an instrument, record what they play on that instrument, create loops, sequence, and a lot more.

By placing instrument creation in VR, users have access to much more expressiveness of movement in their playing. They can place instrument pieces in places that agrees with how they move their body, and how they think each motion should create a sound. They are not limited by the acoustic properties of the instrument, nor are they boxed into the constraints of whoever created the instrument they like the sound of and wish to use. With EXA, they can utilize full range of movement in order to play how they would like to.

Positives:

Virtually Limitless Instrument creation and customization.
Instrument creation and playing in VR utilizes full-body movement.
Can create instruments that work for your body, not learn how to make your body work for a specific instrument - especially in the case of disability.
Recording in EXA is much more physically and visually intuitive. As shown in the video, you can hit 'record' at any time while playing to record the snipped of what you played. Then, you can loop what you recorded for a set time or indefinitely. You can even choose to see other virtual 'players' playing the loop in real-time to better play along with them.
- A sequencer has also been added to EXA to allow for drag and drop mechanics similar to other DAWs.
Can play your instrument wherever you can set up EXA. At this time it is available on HTC Vive and the Oculus Rift.
Can use many different sounds with your custom instruments.
Multiplayer! You can play with other people on local network LAN, or on the internet!

Negatives:

Less physical feedback with your instrument.
No purely acoustic sounds, can only sound as close to a real instrument as a sample of said instrument.
No standardization of playing, so the virtuosity of learning to play a specific instrument is not fully capable.
Only available on tethered VR at the moment, not on more portable options like the Oculus Quest.
- Not as portable as some instruments; for example, a guitar or a clarinet.
Depending on a person's audio setup and computer's processing speed, there may be some audio latency.
Playing with others will have more latency than playing instruments with others in the same room.

EXA In Action

Pre-loaded 'Piano' and 'Drum Kit' Instruments

Drawing Various Ringers

Selecting a Sound for a Ringer

Wheel Instrument

Making an Instrument in VR

The building block of each custom instrument in EXA is the Ringer, a shape that is created and given a specific sound by the user. By placing a number of Ringers in various configurations, players can create any sort of percussion-based instrument they can think of. From the traditional to the abstract, EXA can be tweaked towards the wildest imaginations of any user.

The process to create an instrument is:

Draw a Ringer with the Draw tool
- Oval
- Rectangle
- Triangle
- Straight Line - The only different Ringer. These act more like a guitar string, rather than a simple drum.
Assign a sound to the Ringer you just created.
- EXA comes equipped with hundreds of different sounds, but if you want others you can install Soundfont(SF2) audio packages samped sources such as WAV, AIFF, etc.
- Can be acoustic instrument sounds, synthesized instrument sounds, even specific sound effects.
- After selecting the base sound, you can further customize the sound with loudness, duration of sound, note played, etc.
Scale the Ringer to what size you would like.
Drag the Ringer to wherever you want.
Repeat from step one until the desired instrument is created.
Save the instrument as a bundle.
If more than one instrument is desired at a time, save all the separate instruments together as a Layout.
That's it!

Using EXA To Create an Accessible Instrument

Mary Louise McCord has cerebral palsy, which is mainly classified as a group of movement disorders. She, and many other people with movement disorders don't have the dexterity required by virtually all musical instruments. They can learn how to read music, develop a sense of rhythm, and learn to differentiate notes, but they still cannot play the vast majority of common instruments due to their lack of fine motor control.

Musicians Without Barriers created an instrument for Mary Louise that uses larger arm motions as opposed to finer, precise motions. This allowed her to play the instrument designed for her alongside her other musicians.

Additionally, Damian Mills, the designer of the instrument for Mary Louise, made the decision to place the VR headset behind Mary Louise and monitor in front of her. This allowed Mary Louise to watch the conductor and fellow musicians and still play her custom instrument.

The instrument that Damian made was a large stack of midi triggers laid on top of each other. The large scale allowed Mary Louise to play her instrument with better control because she had better accommodation for error in her arm placement.

Mary Louise

Mary's Instrument

Performance

Page updated

Report abuse