MuSA_RT animates a visual representation of tonal patterns - pitches, chords, key - in music as it is being performed.
MuSA_RT interprets the audio signal received from a microphone to determine pitch names, maintain short term and longterm tonal context trackers, each a Center of Effect (CE), and compute the closest triads (3-note chords) and keys as the music unfolds in performance.
MuSA_RT presents a graphical representations of these tonal entities in context, smoothly rotating the virtual camera to provide an unobstructed view of the current information.
MuSA_RT also offers an experimental Augmented Reality mode when supported.
Start the app and give the app access permission to the microphone when prompted
Tap on "3D" to navigate to the live 3D visualization screen.
Play some music (or sing?): MuSA_RT should start reacting.
To try out the Augmented Reality mode, tap on "AR" (available only if the device supports AR).
Note: MuSA_RT requires a microphone or some audio input device in order to function.
Pitch, triads and keys correspond to locations in the 3D space contained inside the outer spiral (pitches). Pitches along the pitch spiral are separated by a perfect 5th; a given pitch is separated from the pitch directly above by a major 3rd. Major triads are therefore compact upright triangles. Similarly, minor triads are compact triangles. Centers of Effect track the long term (key and spelling) and short term (triad) tonal context.
Select and configure the objects presented in the 3D visualization: pitches, triads, key and associated Center of Effects (CEs).
Enable/disable auto-rotate: when enabled, the model smoothly rotates so that the camera faces the active triad (if any)
The movement time constant determines how fast the rotation occurs (the higher the time constant, the slower the motion)
Select dark/light/auto background mode.
Settings: Spiral Array
Adjust the parameters of the tonal analysis model.
Activation threshold: ignore activation values that are below the threshold (noise)
Time constants for CE computations: lower values make the object more responsive, higher values make the objects more stable. Adapt affects how the object reacts to signal, decay affects how quickly the object fades away in the absence of signal.
For example, to make active triad tracking more responsive, decrease the time constant value for Triad CE Adapt.
Select audio input device, visualize input levels for pitch classes.
Filtering prevents 2 adjacent classes from being active.
Normalization and scaling here are only applied to this visualization, does not affect processing. When normalization is not selected, a slider allows to adjust the scaling applied to the activation values.
MuSA_RT 2.0 Copyright 2021 Alexandre R. J. François.
MuSA_RT 1.0 was originally commissioned by Elaine Chew's MuCoaCo Lab as part of the Music on the Spiral Array . Real Time (MuSA.RT) project, and as companion software to her Springer ORMS Series book Mathematical and Computational Modeling of Tonality: Theory and Applications (2014).
MuSA_RT 2.0 gratefully uses AudioKit v5 for audio input and low level processing (FFT).