CS 8803: Deep Learning for Robotics (Spring 2023)

About the course: Deep learning has emerged as a primary driver of recent advances in robotics research, from learning “end-to-end” sensorimotor skills to supercharging 6D object pose estimation. The new paradigm shifts from traditional feature and model engineering to learning task-relevant representations from raw data. This is fueled by increasingly more affordable hardware and diverse data sources from which algorithms may learn. This course examines how learning approaches have been applied to robotics problems, including 2D/3D perception, tactile sensing, grasping, task and motion planning, and cross-cutting topics such as reinforcement learning, imitation learning, and optimal control. We will also discuss the recent trend of large-scale representation learning for robotics.

Course Information

Lectures: Monday & Wednesday 9:30 - 10:45am in West Architecture 258

Piazza: https://piazza.com/gatech/spring2023/cs8803dlm (Q&A with TAs and Instructors)

Recommended background:

In order to make most out of this course, you should be familiar with deep learning (fair amount of hands-on experience), computer vision (basics of detection, segmentation, camera geometry, etc.), reinforcement learning (Q Learning, actor-critic), and have basic knowledge about robotics (kinematics, configuration space planning, trajectory optimization, etc.).

Recommended courses:

• Machine Learning: Deep Learning (CS 4644/7643) or Machine Learning (CS 7641)

• Perception: Deep Learning (CS 4644/7643) or Computer Vision (CS 6476)

• Robotics: See “Mechanics” and “Control” section of the MS Robotics Curriculum

Grading

This course has no midterm or final exams and is primarily based on student-led presentations and guided discussions. You will be graded on the basis of your class participation and course projects. We will read 1-2 papers for each class, and discuss them in class. Before each lecture, you are expected to submit a short review of the required readings. Each class will also have one or more presenters who are in charge of leading the discussion. Another significant portion of the grade comes from a semester-long project, where you can work in a team of 1-3 people on a research project that is related to the course topics.

The final grade for the course will be tentatively based on the following weights:

• Participation: 50%

  • Paper Review: 15%

  • Paper Presentations: 25%

  • Class Participation: 10%

• Project: 50%

  • Project Proposal: 3%

  • Intermediate Report: 12%

  • Project Presentation: 10%

  • Final Report: 25%

Paper reviews (15% of total grade)

Write reviews for the papers selected for presentation (paper list is in the syllabus below). You are required to complete 10 paper reviews throughout the semester. This accounts for 15% of your total grade. The reviews are due midnight before the presentation day on gradescope (you should have been added to the course).

Refer to this tutorial to learn how to review a Robot Learning paper. 

Paper Presentations (25% of total grade)

We will use the Role Play Seminar presentation format. See the introduction slide (page 35-43) for details & rules. 

Sign up for presentations here.

Syllabus