Background:

Project Daydream is comprised of five students taking Software Design at the Olin College of Engineering. When deciding our final project, we came across this video:

Thus we decided to develop something similar with our final project. This involved working with concepts such as computer vision and object recognition that we had no experience in, and provided us with a unique challenge. Our code takes in a live feed from a camera, recognizes a certain object, and then plays an animation interacting with the object. In our specific case, we used five objects with different colors to test our object recognition, and then overlayed a set of png images for each specific object.

User Instructions:

Users can download our software from our Github repository and run the python file main2.py to launch our software. Our software does require a webcam connected to the computer. After launching our software, just put the objects that can be detected in the view of the webcam and animations would be overladed on them.

Please note that you will need to have the OpenCV library installed for the code to work properly. Click here to download or by typing the following into the terminal:

$ sudo apt-get install python-opencv

This idea, however, would require doing work outside of python and in the android space, of which no one had experience in and deemed it outside the scope of the time we had. Thus we decided to utilize a webcam and have the animations appear on a laptop screen instead of on a phone. In terms of object recognition, we started by looking at OpenCV and Indico for ways to identify and recognize an object. After some initial test with Indico's API, we decided to move on OpenCV because it would require us to train it to recognize all of the objects we wanted. We moved on to using OpenCV and identified different factors, such as contours and colors, to recognize an object. In parallel, we began to test different methods from tkinter to imagemagick to overlay gif animations on top of live video feed, but we were unsuccessful in finding a way to support the gif format. We instead decided to overlay PNG images over each frame of the video. In terms of our current final system, we identify 5 objects based on their color, and then play the animation if the values being read by the feed match those of the range of the trained set.

Attributions:

We would like to thank the entire Software Design spring 2016 teaching team for their guidance over the course of the semester and this final project.