Logistics
Prerequisites:
An intro ML and AI course (e.g. CS188, CS189, CS182) and strong foundations in RL as covered in CS285 (taking simultaneously OK)
Enrollment:
We plan to keep the course small and discussion-oriented. We will favor students pursuing (or actively looking to pursue) research in this area. If you'd like to take the course, please fill out this admissions survey
Weekly Reading Quizzes
To ensure preparedness and help facilitate discussion, there will be weekly reading assignments and a weekly 1-page quiz on the readings. These quizzes will be due every week on Monday at 11:59pm, to be submitted into Gradescope. A reasonable effort demonstrating understanding of the readings will suffice for full credit on each quiz.
Late Reading Quizzes Policy: There will be no slack on these deadlines. However, to provide some flexibility to account for other things in your schedules, we will drop your 3 lowest scoring quizzes.
Scribing, Presenting, Reviewing
Each enrolled student should be signed up on the spreadsheet to scribe for one lecture, present as one of two presenters in a lecture, or review two papers. Please follow the guidelines for scribing, presenting, or reviewing as appropriate.
Homework
There will be 2 Homework assignments, in which you will do a (small) independent, open-ended investigation. Concretely, each homework will have a set of topics to choose from. Within that topic you may be expected to do any of the following (these are just examples): (i) re-implement the algorithm proposed in a paper and study reproducibility of some of the reported experiments; (ii) if a paper has a code release, run this code to first verify results in the paper and then investigate new variations/ablations on the algorithm or investigate how the algorithm fares in new environments; (iii) devise and carry out an experimental comparison of two or more methods that address the same class of problems
Homework can be done in teams of 2 students.
HW1 due Fri 10/1 at 5pm -- topics from Weeks 1-3
HW2 due Fri 10/29 at 5pm -- topics from Weeks 4-7
What to submit: Please submit into Gradescope a 4-page PDF (references can go on an extra page) describing your investigation in a self-contained way. Make sure it has the following sections: (i) Statement of What Idea/Hypothesis is Being Investigated; (ii) Motivation Behind the Investigation; (iii) Description of the Experiments (and/or Theoretical Analysis); (iv) Conclusions and Opportunities for Future Work resulting from your investigation.
Late Homework Policy: Recognizing that students may face unusual circumstances and require some flexibility in the course of the semester, each student will have a total of 7 free late (calendar) days to use as s/he sees fit, but no more than 4 late days can be used on any single assignment. Late days are counted at the granularity of days: e.g., 3 hours late is one late day.
Final Project
The goal of the final project is for you to explore and push the boundaries in learning and decision making. This could be in a variety of ways: e.g. proposal+evaluation of new algorithms / architectures, investigation of an application, benchmarking a range of existing methods, new theoretical results on convergence, sample complexity, etc. We encourage trying to come up with your own project idea. We are also happy to make suggestions and/or brainstorm ideas together. Ideally, the project covers interesting new ground and might be the foundation for a future conference paper submission or product.
Final projects can be done alone, in a group of 2, or in a group of 3. Our expectations will scale linearly with the number of people in the group. We recommend groups of 2 or 3.
Fri Oct 15 at 5pm: Project Proposals Due: 1 page description of project. Submission through google doc to be shared with instructors so we can give feedback/suggestions most easily
Week of Dec 6: Project Presentations
Wed Dec 15 at 11:59pm: Project Reports Due: This should be structured like a 6-page conference paper. I.e., focus on the problem setting, why it matters and what's interesting/novel about it, your approach, your results, analysis of results, limitations, future directions. Cite and briefly survey prior work as appropriate but don't re-write prior work when not directly relevant to understand your approach. References are not counted against the 6 pages.
Late Final Project Policy: No late days can be used for the final project.
Grading
15% weekly quizzes
30% Homework (15% each)
40% Final Project
15% In-Class Presenter (one paper from assigned readings) OR Lecture Scribe for one lecture OR Reviewer for 2 papers (from assigned readings)