mudi

Multimedia Digital Instrument for Composing and Performing Music for Films in Real-time

 

The conception of the MuDI is based on the possibility of using common mobile handheld devices as controllers and musical instruments.

MuDI is a multimedia instrument because it simultaneously (1) diffuses sound, image and video, (2) operates sound and video in real-time, (3) records the sound of the performance, (4) allows to follow not only the film’s images, but also the relationship between the gestures performed and the sound generated (through the projection of an interface), and (5) at the end of the performance to obtain an audiovisual file (composition) in addition to a video score. Furthermore, it allows one to have an effective control over the sound and consequently achieve great musical expression. 

Its system consists of a handheld controller application (running on a PDA) and a computer interface application (projected during the performance).

A method for calibrating the instrument that allowed one to overcome the lack of a reliable position reference point of the accelerometer (the main OSC controller for playing the MuDI) was developed. The reference point obtained through the calibration provide both visual and aural feedback to the composer/performer and it is helpful for playing the instrument.  

Besides the white noise, other digital sound sources, such as a basic circuit of frequency modulation and Our Shepard Sound (OSS) are used [1]. OSS has the characteristic of being an ambiguous sound in terms of Pitch. Probably, we can identify the musical note of the sound, but it will be very difficult to determine the octave it belongs to. This auditory particularity offers new sonic and musical possibilities of listening. 

MuDI was used to compose "sonification - films for music$1" a work premiered in September 2009 in the 12th Brazilian Symposium of Computer Music (SBCM) and “sonification - films for music$2” (2012) in which we use images extracted from the film Metropolis (1927) by Fritz Lang.

The last presentations of MuDI took place in October 2011 in the 2nd International Forum of Musical Itineraries - Music and Gesture, Universidade Nova de Lisboa, Lisbon, Portugal and in May 2012 in the 12th International Conference on New Interfaces for Musical expression (NIME), University of Michigan, Ann Arbor – Michigan, USA.

A promotional video about MuDI at http://www.youtube.com/watch?v=leh-FMwIh5g

"sonification - films for music$2" could be listened at http://www.youtube.com/watch?v=UzYW1cHMFDM

12th International Conference on New Interfaces for Musical expression (NIME) 

[1] More information about OSS could be consulted at https://sites.google.com/site/pp2007pt/uk/auditory-illusion

CONCERTS (videos)

MuDI 1.0 (Played in the 12th SBCM - Recife, Brazil, 2009) 

MuDI 1.1 (Played in the 2nd Forum of Musical Itineraries - Music and Gesture, Lisbon, Portugal, 2011)

home