Pre-requisites:

As such there are no course based pre-requisite requirements of this course. However, basics of data structures and algorithms, probability theory and linear algebra is a much needed pre-requisite to follow the material in this course.

For CS543, there is a prerequisite that either you have already credited CS533 or registered for it in the current semester.

Course Credit Structure:

Both the courses (RL and RL Lab) are 500 level elective courses open to all the departments and all streams of B.Techs, M.Techs and PhDs. The credit structure for the RL course is: 3-0-0-6-3 (3 Credits) and for RL Lab is: 0-0-2-1 (1 Credits). There will be 3 lectures (150 minutes) per week and 2 hrs of lab. Students are expected to invest 6+1 hrs of study time for this course per week. Don't get scared you do have 168 hrs/week to choose your study time from.

Reference Materials:

• Primary textbook - Reinforcement learning: An introduction. Sutton, Richard S., and Andrew G. Barto. 2nd Ed. MIT press, 2018. A draft version (not complete) of this book is available here

• Other reference books/materials

  • Markov Decision Processes by Martin L Puterman, Wiley.

  • Research Papers

  • David Silver's video lectures from Google Deep Mind: https://www.youtube.com/playlist?list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ (2015 Series)

Instructor:

Dr Shashi Shekhar Jha (shashi @iitrpr .ac.in)

Office: #215, 1st Floor, SRB

*We will use google classroom for all course related communications, personal emails to the instructor are discouraged unless the matter needs urgent and personal attention.

Teaching Assistants:

• Mr Armaan Garg (armaan.19csz0002 @iitrpr. ac.in)

• --

Class Communication:

We will use Google Classroom for all course related communications including:

• Class announcements

• Reading material/ slides/ class notes

• Submission of assignments

• Submission of projects/demos

• Attendance

• Quizzes/Exams

To get the classroom code, email the course TAs.

Tentative Grading Policy:

• There will be four exams in the CS533 - RL course:

  • Quiz 1 - 15%

  • Mid-Sem Exam - 20%

  • Quiz 2 - 15%

  • End-Sem Exam - 20%

  • Class Engagement - 10% ( this weightage will be equally shared between CS533 and CS543)

• All exams will be pre-announced. The exact dates of quizzes will be available in the course schedule. Mid-sem and Exam-sem will be held as per the institute time-table. No requests will be entertained for the change in the schedule of quizzes. Further, there will not be any make-up quizzes in the course.

• For the CS543 - RL Lab. course:

  • Lab/Programming Assignments (2) - 20%+20%| There will be 2 programming intensive assignments. Each assignment may span over 2-3 weeks. Students should start working on the assignment as soon as they are announced.

• Course Project /Assignment 3 (have weightage for both CS533 & CS543) :

  • Students can opt to work on a course project (post mid-sem) by presenting a convincing idea to the course Instructor and TAs. A call will be announced in the classroom for the student to submit their ideas. This can be done either individually or in a group of two. The evaluation of the project will be scattered post the mid-sem exam. A final viva voce will be conducted at the end of the course for each project.

  • Students not opting (or not approved) for course project would need to work on a research based assignment i.e. Assignment 3. This assignment has to be done individually as other assignments.

  • Weightage : The course project/assignment 3 will be considered for 20% weightage in CS533 and for 50% weightage in CS534.

This is a tentative breakup of the grades and can change at the discretion of the Instructor. However, any change in grading policy will be duly intimated in advance.

• Getting a Pass Grade

In order to successfully clear the course, a student is expected to secure at least 40% of the total weightage in both the courses CS533 and CS534. If a student clears one course and fails in the other, s/he will be awarded an 'E' grade in both the courses.

• Course Audit

For those who are auditing the course (only CS533 auditing is allowed), they need to secure at least 40% of the total weightage (except the course project) in order to get a pass grade.

Academic Integrity and Honour Code:

• Any kind of plagiarism/cheating/copying etc. will attract an F grade in the course.

• Students are advised to read and understand about plagiarism and never indulge in the same.

• Students are encouraged to discuss and seek guidance (if needed) without breaking any academic integrity.

• All submissions ought to be the original work of the students.

• Any external source must be properly cited mentioning proper references.