Topics and Interests
Vision-to-Audio Translation for Dynamic Environments: Leveraging visual depth, scene representations such as neural radiance fields and 3D Gaussian splatting, and material cue to predict acoustic parameters and reverberation characteristics.
Audio as a Spatial Condition for Visual and Social Presence: Utilizing sound localization and speech semantics to drive avatar behavior, direct multi-user gaze, guide visual attention beyond the field of view, and enhance conversational presence.
Mobile Sensing and Egocentric Multimodal Understanding: Integrating on-device sensing, IMU-based motion tracking and spatial sensor data across mobile, wearable, and peripheral devices for multimodal sensor fusion, behavioral tracking, spatial understanding, and embodied interaction under constrained computational budgets.
Interactive Digital Twins and Multi-User Social XR: Spatial audio as an amplifier of co-presence in multi-user environments where remote spaces, not just avatars, are augmented into the local space, closing the presence gap left by occlusion bound visual rendering through omnidirectional, spatially surrounding sound which understands depth cues.
Dataset Interoperability: Defining unified data structures, metadata, and exchange formats that allow calibrated visual, acoustic, and mobile sensor recordings to be integrated and reused across modalities without dataset-specific conversion.
Workshop Contents
Keynote Talks: Four invited talks, one from each of the workshop’s core disciplines: spatial audio, computer vision, mobile sensing, and HCI.
Live Demos: Interactive spatial audio demonstrations by the organizers and authors of accepted demo papers.
Group Discussion: Cross-disciplinary groups discuss practical constraints such as latency budgets, hardware limits, and data interoperability. The organizers will support groups that commit to follow-up work.
Target Audience
We anticipate a diverse group of approximately 50 participants, comprising academic researchers, industry practitioners, and students whose work intersects with computer vision, audio signal processing, and HCI.
Specifically, this workshop is designed for:
XR and Spatial Audio Researchers: Those bridging the gap between visual scene understanding and spatial audio rendering in real-time interactive systems.
Multimodal System Developers: Practitioners tackling hardware, sensing, and computational constraints in embodied virtual environments.
Social and Multi-User XR Engineers: Developers focusing on avatar interaction, spatial communication, and audio-visual presence.