Computer Vision

Abstract

Computer vision is becoming an increasingly important tool for modern robotics. In this project you will be exploring a computer vision algorithm in depth with a team of 1 or 2 other students. You will have quite a bit of autonomy to choose a project topic that interests you.

Teaming

For this project, you are required to work with at least one other student. There is a strict maximum of three members per team.

Project Topic

Your project should be about computer vision and its intersection with mobile robotics. While it is likely that a substantial component of your project will be about the vision part (and not necessarily its application to mobile robots), you should scope your project so that you have time to deploy your system on the Neatos.

In your project proposal you will be coming up with an implementation plan. That is, if you are using a particular algorithm to solve a problem, which parts of the algorithm will you implement, and which will you use pre-built implementations for? Be strategic in these decisions to balance learning about algorithms with programming the Neatos to do something interesting. As a brief aside about the pedagogy of this course, since as the course goes on I am going to spend less time discussing algorithms in the structured portion of the class, it is important for you to do project-based exploration of algorithms. To this end, I will expect that you will implement a non-trivial portion of the algorithms you wind up using.

Suggested Algorithm Topics

    1. Object tracking
  1. Image segmentation
    1. Object detection
  2. Text recognition
    1. Fiducial tracking
    2. Visual odometry
    3. Structure from motion

(see attached files at the end of of this page for some resources)

Deliverables

There are four deliverables for this project.

Project Proposal

Please come to class on Day 14 having thought of some project ideas. You should make sure to develop these ideas to some extent before coming to class. You don't have to be at the full project proposal stage, but you should be further than the "bunch of ideas on post-it notes" stage. Here is a potential structure you could use for fleshing out each idea. I'd try to come to class with at least two ideas fleshed out in this manner, but you should have at least have one.

    • Project title
    • What's the elevator pitch? What do you want the robot to do with computer vision, and why is this an interesting / compelling project.
    • What algorithms will you explore through this project? Which algorithms do you imagine implementing and which do you think you will grab from pre-built packages.
    • What do you hope to learn through this project?
    • Any other considerations (e.g., work schedules, work style compatibility) that someone should think about before working on this project?

Make sure to come to class with your idea on paper. This is crucial for Day 14 when we'll do some affinity diagramming where we read each others' ideas and group together similar proposals.

24 hours after class on Day 14 (5pm on March 7th for the M/R section and 12:30 on March 8th for the T/F section) you should share a Google doc with me (paullundyruvolo@gmail.com) with a project proposal. Please include the answers to the following questions. You should include enough detail for me to be able to give you minimum feedback, DO NOT just include one sentence responses to each of these prompts.

  • Who is on your team?
  • What is the main idea of your project?
  • What are your learning goals for this project?
  • What algorithms or computer vision areas will you be exploring?
  • What components of the algorithm will you implement yourself, which will you use built-in code for? Why?
  • What is your MVP?
  • What is a stretch goal?
  • What do you view as the biggest risks to you being successful (where success means achieving your learning goals) on this project?
  • What might you need from me for you to be successful on this project?

In-class Presentation / Demo

I'd like each team to spend about 10 minutes presenting what they did for this project. You can structure the presentation in whatever manner you'd like, however, you should try to meet these goals:

  • Explain the goal of your project
  • At a high-level explain how your system works
  • Demonstrate your system in action (either in a video [recommended] or live). If your system doesn't work completely yet, that is fine, try to show at least one component of your system in action.

This presentation / demo should be very informal. This presentation will be assessed in a purely binary fashion (basically did you do the things above).

Code

You should turn in your code and writeup via Github. Please fork your repo from this one.

Writeup

in your ROS package create a file to hold your project writeup. Any format is fine (markdown, word, pdf, etc.). Your writeup should touch on the following topics:

  • What was the goal of your project?
  • Describe how your system works. Make sure to include the basic components and algorithms that comprise your project.
  • Describe a design decision you had to make when working on your project and what you ultimately did (and why)? These design decisions could be particular choices for how you implemented some part of an algorithm or perhaps a decision regarding which of two external packages to use in your project.
  • How did you structure your code?
  • What if any challenges did you face along the way?
  • What would you do to improve your project if you had more time?
  • Did you learn any interesting lessons for future robotic programming projects? These could relate to working on robotics projects in teams, working on more open-ended (and longer term) problems, or any other relevant topic.

Useful Resources