Time Tuesday and Thursday, 12:30 - 1:50 PM EST
Room NSH 3002
Instructor David Held (Office hours upon request; send a Piazza message to schedule a time!)
TA: Ben Eisner (Office hours: Mondays 1-2 PM, NSH 4125)
Instructor+TA Email List 16881-tas@lists.andrew.cmu.edu
Course Discussion / Announcements: https://piazza.com/cmu/spring2023/16881 (make sure to sign up to receive announcements and to submit assignments)
We encourage you to ask questions and have discussions on Piazza; you can even do so anonymously.
Deep RL has a lot of promise to teach robots how to choose actions to optimize sequential decision-making problems, but how can we make deep RL work in the real world?
This is a seminar course in which we read papers related to deep learning for robotics and analyze the tradeoffs between different approaches. We will read mostly state-of-the-art papers that were very recently published (e.g. this year’s CoRL and RSS), but we will also look at some older papers that use different approaches. The goals of the course are to 1) understand what is needed to make deep learning work for robotics 2) analyze the tradeoffs between different approaches.
Each class, 2 papers will be presented. These papers will both achieve a similar robotics task but will use different learning-based approaches. The class will discuss these papers and try to understand the strengths and limitations of the approach described in each paper. The list of papers that we will be discussing can be found here: https://sites.google.com/view/16-881-cmu/paper-lists?authuser=0
Students will also work on a class project related to robot learning of their choosing.
The seminar is a great followup course to 16-831, 16-884, 10-403, or 10-703.
The timing for the class will be as follows:
23 minutes for presentation 1 (+4 minutes for discussion and changeover)
23 minutes for presentation 2 (+4 minutes for discussion)
10 minute small group discussion
16 minute class discussion
Students are expected to have already have a basic understanding of reinforcement learning, such as from 16-831, 16-884, 10-403, or 10-703 or a similar course, prior to taking this course. If you are not sure if your background is sufficient, please post on Piazza to ask. If you need a refesher, I highly recommend Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto, especially Chapters 2-6 and 13. Also, the second lecture of the course will cover a quick review of reinforcement learning.
Presentations: 30%
Paper reviews: 40%
Class project: 30%
Become familiar with some recent papers on deep reinforcement learning for robotics
Think about the tradeoffs between different approaches for solving a robotics problem
Understand what is needed to make deep learning work for robotics
Improve critical reading and analysis skills
Improve presentation skills
You are encouraged to work together and discuss the papers with others BUT you must write up your own paper reviews. If we find highly identical reviews without proper accreditation of collaborators, we will take action according to university policies.
Studies have shown that your mental health can benefit by maintaining a healthy lifestyle by eating well, exercising, avoiding drugs and alcohol, getting enough sleep and taking some time to relax. These practices can help you achieve your goals and cope with stress.
Even with these practices, it is natural to struggle with mental health challenges. All of us benefit from support during times of struggle. You are not alone. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is often helpful. If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at http://www.cmu.edu/counseling/. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.
All people have the right to be addressed and referred to in accordance with their personal identity. Students are encouraged to use NameCoach to indicate the name that they prefer to be called and to identify pronouns with which they would like to be addressed. I will do my best to address and refer to all students accordingly and encourage everyone in the class to do so as well.
We must treat every individual with respect. We are diverse in many ways, and this diversity is fundamental to building and maintaining an equitable and inclusive campus community. Diversity can refer to multiple ways that we identify ourselves, including but not limited to race, color, national origin, language, sex, disability, age, sexual orientation, gender identity, religion, creed, ancestry, belief, veteran status, pregnancy, or genetic information. Each of these diverse identities shape the perspectives our students, faculty, and staff bring to our campus. We at CMU will work to promote diversity, equity and inclusion. We acknowledge our imperfections while we also fully commit to the work, inside and outside of our classrooms, of building and sustaining a campus community that increasingly embraces these core values.
Each of us is responsible for creating a safer, more inclusive environment. Unfortunately incidents of bias or discrimination do occur, whether intentional or unintentional. They contribute to creating an unwelcoming environment for individuals and groups at the university. Therefore, the university encourages anyone who experiences or observes unfair or hostile treatment on the basis of identity to speak out for justice and support, within the moment of the incident or after the incident has passed. Anyone can share these experiences using the following resources:
Center for Student Diversity and Inclusion: csdi@andrew.cmu.edu, (412) 268-2150
Report-It online anonymous reporting platform: www.reportit.net username: tartans password: plaid
All reports will be documented and deliberated to determine if there should be any following actions. Regardless of incident type, the university will use all shared experiences to transform our campus climate to be more equitable and just.
The course instructors also encourage anyone who experiences or observes unfair or hostile treatment to report such incidences to the course instructors so that we can address such situations.
If you are worried about affording food or feeling insecure about food, there are resources on campus who can help. Email the CMU Food Pantry Coordinator to schedule an appointment:
Pantry Coordinator
cmu-pantry@andrew.cmu.edu
412-268-8704 (SLICE office)
If you have a disability and are registered with the Office of Disability Resources, I encourage you to use their online system to notify me of your accommodations and discuss your needs with me as early in the semester as possible. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, I encourage you to contact them at access@andrew.cmu.edu.
It is my intent that students from all diverse backgrounds and perspectives be well served by this course, that students’ learning needs be addressed both in and out of class. It is my intent to present materials and activities that are respectful of diversity: gender, sexuality, disability, age, socioeconomic status, ethnicity, race, and culture. Your suggestions are encouraged and appreciated. Please let me know ways to improve the effectiveness of the course for you personally or for other students or student groups. In addition, if any of our class meetings conflict with your religious events, please let me know so that we can make arrangements for you.
If you need extra help for the course beyond what the instructor and TAs can provide, you can also use the Student Academic Success Center (SASC).
Academic Coaching--This program provides holistic, one-on-one peer support and group workshops to help undergraduate and graduate students implement habits for success. Academic Coaching assists students with time management, productive learning and study habits, organization, stress management, and other skills. Request an initial consultation here.
Peer Tutoring--Peer Tutoring is offered in two formats for students seeking support related to their coursework. Drop-In tutoring targets our highest demand courses through regularly scheduled open tutoring sessions during the fall and spring semesters. Tutoring by appointment consists of ongoing individualized and small group sessions.You can utilize tutoring to discuss course related content, clarify and ask questions, and work through practice problems. Visit the webpage to see courses currently being supported by Peer Tutoring.