About the Course:

Reinforcement Learning (RL) forms one of the core areas of artificial intelligence with relevance to a variety of domains that include robotics, game theory, control theory, multi-agent systems, swarm intelligence, and operations research. This course will provide a mathematical foundation for the field of reinforcement learning along with the core challenges and approaches, including generalization and exploration. Various latest advancements such as DeepRL, InverseRL, etc. will also be covered in this course. The course will equip students with advanced skillset in the general area of artificial intelligence and prepare them to advance their careers (both research and professional) in the field artificial intelligence.

Pre-requisutes:

As such there are no course based pre-requisite requirements of this course. However, basics of data structures and algorithms, probability theory and linear algebra is a much needed pre-requisite to follow the material in this course.

For CS543, there is a prerequisite that either you have already credited CS533 or registered for it in the current semester.

Course Credit Structure:

Both the courses (RL and RL Lab) are 500 level elective courses open to all the departments and all streams of B.Techs, M.Techs and PhDs. The credit structure for the RL course is: 3-0-0-6-3 (3 Credits) and for RL Lab is: 0-0-2-1 (1 Credits). There will be 3 lectures (150 minutes) per week and 2 hrs of lab. Students are expected to invest 6+1 hrs of study time for this course per week. Don't get scared you do have 168 hrs/week to choose your study time from.

Reference Materials:

• Primary textbook - Reinforcement learning: An introduction. Sutton, Richard S., and Andrew G. Barto. 2nd Ed. MIT press, 2018. A draft version (not complete) of this book is available here

• Other reference books/materials

  • Markov Decision Processes by Martin L Puterman, Wiley.

  • Research Papers

  • David Silver's video lectures from Google Deep Mind: https://www.youtube.com/playlist?list=PLqYmG7hTraZBiG_XpjnPrSNw-1XQaM_gB (2015 Series)

Instructor:

Dr Shashi Shekhar Jha (shashi @iitrpr .ac.in)

Office: #215, 1st Floor, SRB

*We will use google classroom for all course related communications, personal emails to the instructor are discouraged unless the matter needs personal attention.

Teaching Assistants:

• Mr Armaan Garg (armaan.19csz0002 @iitrpr. ac.in)

• Ms Surbhi Madan (surbhi.19csz0011 @iitrpr. ac.in)

Class Communication:

We will use Google Classroom for all course related communications including:

• Class annoucements

• Reading material/ slides/ class notes

• Submission of assignments

• Submission of projects/demos

• Attendance

• Online Quizzes/Exams

To get the classroom code, email the course TAs.

Tentative Grading Policy:

• There will be four equally balanced exams for the CS533 - RL course:

  • Quiz 1 - 20%

  • Mid-Sem Exam - 20%

  • Quiz 2 - 20%

  • End-Sem Exam - 20%

  • Class Engagment - 5%

    • All exams will be pre-annouced. The exact dates of quizzes will be available in the course schedule. Mid-sem and Exam-sem will be held as per the institute time-table. No requests will be entertained for the change in the schedule of quizzes. Further, there will not be any make-up quizzes in the course.

• For the CS543 - RL Lab. course:

  • Lab/Programming Assignments (2) - 20%+20%| There will be 2 programming intensive assignments. Each assignment may span over 2-3 weeks. Students should start working on the assignment as soon as they are annouced.

• Course Project (have weightage for both CS533 & CS543) :

  • Weightage : 15%(cs533)+60%(cs534) = 75% |Each student has to take up a implementation project on a particular topic of interest in RL. The project can be done either individually or in a group of two students. The evaluation of the project will be scattered post the mid-sem exam. The final viva voce will be conducted at the end of the couse for each project.

This is a tentative breakup of the grades and can change at the discretion of the instructor. However, any change in grading policy will be duly intimated in advance.

In order to successfully clear the course, a student is expected to secure at least 40% of the total weightage in both the courses CS533 and CS534. If a student clears one course and fails in the other, s/he will be awarded an 'E' grade in both the courses.

For those who are auditing the course, they need to secure at least 40% of the total weightage (except the course project) in order to get a pass grade.

Academic Integrity and Honour Code:

• Any kind of plagiarism/cheating/copying etc. will attract an F grade in the course.

• Students are advised to read and understand about plagiarism and never indulge in the same.

• Students are encouraged to discuss and seek guidance (if needed) without breaking any academic integrity.

• All submissions ought to be the original work of the students.

• Any external source must be properly cited mentioning proper references.

Counselling Support:

• Any student experiencing any mental or emotional stress can seek the free and confidential clinical counselling by contacting the Counselling Cell of IIT Ropar.

  • Deepak Kr. Phogat (Clinical Psychologist, Counselor)

    • Common Office: 01, Medical Centre, Utility Block, Main Campus

    • Email: deepak.phogat @ iitrpr ac in

    • Phone: 01881-24-2264

  • Bhawna Suri (Counseling Psychologist, Counselor)

    • Email: bhawna @ iitrpr ac in

    • Phone: 01881-24-2261