—— An Interactive Installation for Reconstructing Identity Through Tangible Interaction and Projected Space
Cindy Jin
ICAM Senior Project 2026
Fragmented Room is an interactive installation that explores how memory and identity can be reconstructed through fragmented space and embodied interaction. The installation combines four miniature physical rooms with a projected digital environment, inviting participants to rebuild a fragmented narrative by returning puzzle pieces to their corresponding locations.
Each puzzle piece contains an embedded magnet that activates a reed switch beneath the puzzle board when correctly placed. These signals are transmitted to a real-time TouchDesigner system, where corresponding AI-generated visual elements gradually appear within the projected environment. As participants reconstruct the physical rooms, the digital scene simultaneously unfolds, transforming puzzle assembly into a process of spatial storytelling.
Beyond physical interaction, the installation also incorporates gesture-based exploration through MediaPipe hand tracking. Participants can rotate the projected digital environment using a simple hand gesture, encouraging the reconstructed space to be viewed from multiple perspectives. Rather than presenting a fixed narrative, Fragmented Room invites audiences to actively reconstruct memory and identity through movement, interaction, and perception.
Fragmented Room investigates how identity can be represented through space rather than through direct narrative exposition. Inspired by the idea that memories are incomplete, fragmented, and continuously reconstructed, the project transforms physical reconstruction into a metaphor for psychological reconstruction. Participants gradually rebuild both the physical and projected environments, allowing identity to emerge through interaction rather than explicit storytelling.
The installation is also influenced by interactive media artworks that distribute images and narratives across physical space, encouraging audiences to construct meaning through movement and participation. Instead of presenting a complete environment from the outset, Fragmented Room reveals fragmented objects and memories over time, emphasizing that identity is not fixed but continuously reshaped through experience and perception.
By combining tangible interaction, projected digital imagery, and gesture-based exploration, the project encourages visitors to become active participants rather than passive observers. The physical act of assembling puzzle pieces parallels the ongoing process through which people reconstruct memories, relationships, and self-perception, positioning spatial interaction as both a storytelling strategy and a reflection on identity itself.
The installation integrates physical computing, projected digital media, and computer vision into a unified interactive system. Four miniature rooms contain removable puzzle pieces embedded with magnets, while reed switches installed beneath the puzzle board detect their placement. An Arduino Uno continuously monitors the sensor states and transmits the data to TouchDesigner through serial communication, allowing physical interactions to directly control the projected digital environment.
Rather than using traditional 3D models, the projected scene is composed of AI-generated visual assets that are layered and managed within TouchDesigner. Each puzzle piece corresponds to a specific visual element that gradually appears as participants reconstruct the physical installation, creating a synchronized relationship between tangible interaction and digital visualization.
To further encourage exploration, the installation incorporates MediaPipe hand tracking as a secondary interaction layer. By performing a simple fist-closing and opening gesture, participants can rotate the projected environment by 90 degrees, allowing the reconstructed space to be viewed from different perspectives. Together, the physical interface, sensing hardware, real-time rendering system, and gesture recognition establish an embodied interaction loop that connects movement, perception, and narrative reconstruction.
Brainstorm final project idea and write initial project proposal
Create a detailed project timeline
Create a minimal Blender prototype including a simple room model and basic objects, and order Arduino components
Build a basic Arduino prototype that reads input and sends serial data to a computer, and test Blender to TouchDesigner compatibility as a proof of concept
Brainstorm physical puzzle piece concepts and narratives
Start 3D modeling the physical puzzle pieces scene
Finish one scene and 3D print
Assemble 3D printed parts and Arduino
Complete final prototype with one finished scene
Tech Rider (space & equipment setup)
Finish room 2 design (bathroom)
Finish room 3 design (theater)
Finish room 4 design (therapy room)
Finish 3D printing and painting all 4 physical scene
Revisit Touchdesigner and do some finishing touches, try projection results
Finished digital scene
Connect Arduino with all of the rooms
Finished wiring & soldering
Gallery installation and exhibition
Email: yajin@ucsd.edu