Reinforcement Learning (RL) forms one of the core areas of artificial intelligence with relevance to a variety of domains that include robotics, game theory, control theory, multi-agent systems, swarm intelligence, and operations research. This advanced course provides an in-depth exploration of RL, from its seminal works to the most recent state-of-the-art developments in the field. We will cover a wide range of modern topics, including: Exploration, Model-based, Offline, Imitation Learning, Safe RL, Inverse RL, Meta RL, Transfer RL, etc.
The course is research-oriented, and students are expected to engage deeply with the literature, culminating in the creation of an original research paper on a relevant topic of their choice.
As such, there are no course-based pre-requisite requirements for this course. However, proficiency in the following would be needed to understand and follow the material of the course along with completing the project:
Data Structures and Linear Algebra: You must understand basic data structure concepts. You should be comfortable taking (multivariable) derivatives and know matrix/vector notations and operations.
Probability Basics: Familiarity with basic probability distributions (Continuous, Gaussian, Bernoulli, etc.) is expected. Further concepts such as Expectation, independence, probability distribution functions, and cumulative distribution functions must be clear to work-out the material in this course.
Python coding: All projects will involve coding in python. Hence, familiarity with the python programming language is required. They may also require the use of deep-learning libraries such as TensorFlow, Keras, Pytorch, and JAX.
Knowledge of CV / ANN / ML / DLCV / AI/ RL / Game Theory / FDS.
This course is a 700 level elective courses open to all the departments and all streams of B.Techs, M.Techs/M.Sc. and PhDs. The credit structure for the RL course is: 2-0-2-5-3 (3 Credits) . There will be 1 lecture (150 minutes) per week.
Primary textbook - Reinforcement learning: An introduction. Sutton, Richard S., and Andrew G. Barto. 2nd Ed. MIT press, 2018. A draft version (not complete) of this book is available here
Other reference books/materials
Markov Decision Processes by Martin L Puterman, Wiley.
Reinforcement Learning and Optimal Control by D. P. Bertsekas, Athena Scientific.
Research Papers.
David Silver's video lectures from Google Deep Mind: https://www.youtube.com/playlist?list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ (2015 Series)
Deep Mind 2021 Reinforcement Learning Lecture Series: https://www.deepmind.com/learning-resources/reinforcement-learning-lecture-series-2021
OpenAI Spinning up in Deep RL - https://spinningup.openai.com/en/latest/
NPTEL course on RL by Prof B. Ravindran - https://nptel.ac.in/courses/106106143
Dr Shashi Shekhar Jha (shashi @iitrpr .ac.in)
Office: #215, 1st Floor, SRB
*We will use google classroom for all course related communications, personal emails to the instructor are discouraged unless the matter needs urgent and personal attention.
Mr Rahul Narava (syam.21csz0018@ iitrpr.ac.in )
Classroom Room : TBD
Tue - 4 PM
We will use Google Classroom for all course related communications including:
Class announcements
Reading material/ slides
Attendance
Project Evaluations
Submission of projects/demos
To get the classroom code, email the course TA.
There will be No exams in the CS733 - Topics in RL course:
Paper Presentations - 30%
Class Engagement - 20%
Project - 50%