CS 8803: Deep Learning for Robotics (Spring 2024)

About the course: Deep learning has emerged as a primary driver of recent advances in robotics research, from learning “end-to-end” sensorimotor skills to supercharging 6D object pose estimation. The new paradigm shifts from traditional feature and model engineering to learning task-relevant representations from raw data. This is fueled by increasingly more affordable hardware and diverse data sources from which algorithms may learn. This course examines how learning approaches have been applied to robotics problems, including 2D/3D perception, tactile sensing, grasping, task and motion planning, and cross-cutting topics such as reinforcement learning, imitation learning, and optimal control. We will also discuss the recent trend of large-scale representation learning for robotics.

Course Information

Lectures: Monday & Wednesday 9:30 - 10:45am in West Architecture 258

Piazza: https://piazza.com/gatech/spring2024/cs8803dlm (signup code available on Canvas)

Office hours: Check for regular office hours on calendar below. 

For extra office hours, please email the instructor / TAs you would like to meet with.

Recommended background:

In order to make most out of this course, you should be familiar with deep learning (fair amount of hands-on experience), computer vision (basics of detection, segmentation, camera geometry, etc.), reinforcement learning (Q Learning, actor-critic), and have basic knowledge about robotics (kinematics, configuration space planning, trajectory optimization, etc.).

Recommended courses:

• Machine Learning: Deep Learning (CS 4644/7643) or Machine Learning (CS 7641)

• Perception: Deep Learning (CS 4644/7643) or Computer Vision (CS 6476)

• Robotics: See “Mechanics” and “Control” section of the MS Robotics Curriculum

Grading

This course has no midterm or final exams and is primarily based on student-led presentations and guided discussions. You will be graded on the basis of your class participation and course projects. We will read 1-2 papers for each class, and discuss them in class. Before each lecture, you are expected to submit a short review of the required readings. Each class will also have one or more presenters who are in charge of leading the discussion. Another significant portion of the grade comes from a semester-long project, where you can work in a team of 1-3 people on a research project that is related to the course topics.

The final grade for the course will be tentatively based on the following weights:

• Participation: 40%

  • Paper Review: 10%

  • Paper Presentations: 20%

  • Class Participation: 10%

• Assignment: 20%

  • Two homework assignments: programming + short questions

• Project: 40%

  • Project Proposal and Presentation: 2%

  • Intermediate Reports (2x): 8%

  • Final Project Presentation: 10%

  • Final Report: 20%

All project and assignments have hard deadlines. Late submission will receive a zero.

Class participation (10% of total grade)

We will take attendance for 12 randomly-selected classes. You will get penalized if you miss more than 2 attendance-taking classes. 

Paper reviews (10% of total grade)

Write reviews for the papers selected for presentation (paper list is in the syllabus below). You are required to complete 10 paper reviews throughout the semester. This accounts for 10% of your total grade. The reviews are due midnight before the presentation day on Canvas (you should have been added to the course).

Refer to this tutorial to learn how to review a Robot Learning paper. 

Paper Presentations (20% of total grade)

We will use the Role Play Seminar presentation format. See this doc for instructions.

Signup sheet available on piazza.

Course Project (40% of total grade)

Project Proposal: Instruction

Intermediate Project Reports: Report 1, Report 2

Final Presentation: Instruction

Final Report: Instruction

Example final report from previous semester: Google Drive

"Default" project ideas: doc