This interactive installation uses Kinect and Unity to create a fully immersive experience. User movements are captured and translated into visual and auditory feedback, creating a magical experience. Visual effects are projected onto three walls and the floor with two powerful projectors, immersing users in a new world. The journey begins with an enchanting display of northern lights, drawing users in.
As users move, they interact with a "Spirit" and trigger particle effects, leaving trails of growing plants and clover footprints on the ground. Surround sound enhances the sensory experience. The installation is mobile, designed for easy setup in various locations, offering a captivating and interactive adventure.
I initiated this project and played a key role in shaping the overall vision and user experience of Spirit Garden, often also acting as a group leader and mediating the group's different ideas.
My main task was coding and working with Unity. I focused on coding essential features, including:
Developing C# scripts to control plant growth and variations in appearance.
Implementing logic for interactive transitions between idle (Aurora display) and active modes (user interaction with the "Spirit").
Exploring Unity's VFX Graph for designing particle systems representing the Spirit.
I also contributed to animating the plant growth using Blender and integrating these animations into Unity as Signed Distance Fields (SDFs).
We worked with Kinect v2 and utilized Rumen Filkov’s "Kinect v2 Examples with MS-SDK" asset for interaction scripts. Despite challenges like tracking accuracy, particularly for lower body joints, the system successfully captured hand joint movements to provide visual feedback.
We integrated Unity with projection mapping software (Pixera and Vioso6) to align visuals across three walls and the floor. This setup was complex but essential for creating a seamless immersive experience. For our prototype we had to drop the floor projection during development.
The installation featured a custom soundscape created with FL Studio, Pro Tools, and Max MSP, enhancing the sensory immersion with spatialized audio linked to user interactions.
In August 2023, Spirit Garden debuted at the Linachtalsperre in Vöhrenbach during the Reservoir festival, allowing users to engage in an interactive, meditative journey through creativity and nature.