Lab2 - OpenFramework

Motion Painting  by Tamara & Grace

Watching a dancer perform, it feels like she is using her whole body to paint a beautiful picture in the air.

How about recording the dancer's moves as a real painting? and how will that look like?

no... not this one. This:

The "Motion Painting" system allows users to draw on the  computer screen by moving their body in front of the computer camera. (The first picture is kinda irrelevant :P just a cool project I found during our development)

Function description: 

    The system has two painting modes: one using a paint brush that draws lines.  Users' body movements will control the brush moving left right up or down (relative positioning). The other mode is without a certain paint brush, but wherever your body moves the screen will be painted at the reflected position through webcam (absolute positioning). Users can switch between modes by pressing "o" on the keyboard. Pressing "q" can clean up the screen, and "v" can show or hide the camera scene. Users can always change their paint color by trying to move the little "cursor" to the "color circle" to acquire its current color. (Actually not only body movements, any movements within the webcam area can be detected.)

What we basically used are:  

- webcam

- xcode for Mac and vs2010 on windowns (which caused a lot of trouble and doesn't really work very well...)

- openFramework library and several addons related to motion tracking

This program is based on a project called "motionPong" developed by Tangible interaction

The following picture shows the general structure between classes of this program:

Some developing details:

During the development we encountered many problems. 

The first one is to get OF examples working on the computer. The beginning is always hard. When I finally was able to get SOME examples running,a  new problem was to get the downloaded addons and projects working. Those "No such file or directory" "unresolved external symbols" and all the other errors gave me a hard time when I simply just want to see some example projects. Well, finally errors are removed one by one.

OpenFramework is new for us. So even something as simple as trying to keep the drawing on the screen (since the camera video is updating the screen all the time) can be time-consuming to tackle. We tried to search for a better way but didn't really find anything useful so the only way we could think of is to store every line and point that has been drawn and redraw them all the time. 

The color changing is interesting, and also tricky. The "color circle" is actually the ball in the original pong game. We thought that acquiring color from that ball would be nice but when we tried it we found that whenever you acquire a new color, since you're redrawing everything all the time so what you have drawn before would also change to current color. That means we need to not only store the coordinates but also store the color information. We used List for storing all those data and we created several objects that could store both coordinates and colors.

Current problem:

According to our user evaluation, the paint brush and cursor is still a bit hard to control, and not accurate. 

As time is limited, we could only have this prototype. But our whole concept is to make it more accurate and can beautifully track down the movements and present it with different colors. Maybe we can optimize it later on and use it with real dancers at a live performance! :)

For more project pictures please check here.

For code and demo video, please refer to Tamara's page. :)