Research
The project draws on the research areas shown below. The stage of advancement for each item is indicated between the brackets [].
3D audiovisual musical representations [finished]
Utilisation of 3D game engines for music creation and performance
Musical Ensemble Synchronization [in-progress]
Bi-directional multi-channel audiovisual streaming with system-wide synchronization.
Video-based conductor feed for remote ensemble synchronization.
3D audio [early]
Capture and Rendering of 6-DoF navigable audio scenes and acoustic spaces (parametric or geometric).
Acoustic space modeling and simulation
AR-based rendering clients
Remote audio rendering [planned]
Scalable solutions for user-specific remote audio rendering of scenes with many real-time audio input streams.
Networked-based distribution of live music performance events [in-progress]
For large shared AV perspective venues, such as the Satosphere, with multiple bi-directional audio streams.
For single-user AR-based user experiences involving remotely rendered one-way stereo (binaural) audio streams, and renderer-bound 6-DoF perspective updates.