How it works!

I focused on creating a real-time, interactive drawing experience using hand-tracking technology (Borrowed from Ml5.js!). The core of the system is structured around a few global states: showIntro, showVideo, and NowStartDrawing. They control the visual transitions between introduction, video grid display, and interactive drawing. Rather than using class-based organization, the logic is handled procedurally, but modularly through conditional checks and phased transitions in the draw() loop. I used this method since I found it easier to organize my thoughts and outcome of the project. I also did something similar, with procedural modules for the final project in Interaction Lab. 

The central interaction—pinching fingers together to draw or clear the screen—is done using ML5’s handPose model. By analyzing distances between specific hand keypoints (thumb and index for drawing; thumb and middle for clearing), the code detects gestures and maps them to expressive visual behaviors.

The drawing behavior is realized through storing points in a drawingPoints array, which is updated every frame based on pinch detection. At first, I wasn't sure how to do this since the background updates every frame. But from the help of youtube and AI generated examples, I was able to go past this obstacle. These points are rendered as colored circles on the canvas. To emphasize the artistic theme, the video feed is stylized with a dynamic grid distortion effect (provided by Professor Moon!) that visually abstracts the camera input while preserving a sense of presence.

Area for Improvement

Right now the prototype is a bit slow since it uses a lot technology that uses a lot of memory. So that will be the first improvement. Also adding colors to every circle generated, with the option for randomized radiuses on the circles will be nice. Since the theme of the lines is supposed to represent a notebook, its also helpful for the user if I added instructions as "notes" on the sheet of paper. Final some sound effects would be a nice touch if there is room before the IMA Show. But overall, I'm the most worried about the "laggyness" of the program.

Here are some pictures of the prototype's prototype and the prototype...

I wanted to emphasize the "YOU" in the title with the prototype's prototype

Very Freaky!