Five Day Workshop on Artificial Intelligence

Indian Institute of Information Technology, Vadodara

02-06 December, 2019

Sponsored By

AICTE Training and Learning (ATAL) Academy

The use of computer to solve complex problems is the fundamental theme of Artificial Intelligence. The notion of intelligence being captured by problem solving ability reflects throughout the course. Understanding the difficult problems in computation and interpreting softwares as intelligent agents is important. Modeling the problems in a way that can be solved using computer programs is very crucial to understanding artificial intelligence. Reinforcement learning (RL) is a computational approach to understanding and automating goal-directed learning and decision making. The course addresses the computational issues that arise when learning from interaction with an environment in order to achieve long-term goals. RL uses the formal framework of Markov decision processes to define the interaction between a learning agent and its environment in terms of states, actions, and rewards.

About the Institute

Indian Institute of Information Technology Vadodara was established in 2013 under Public Private Partnership of Government of India, Government of Gujarat, Tata Consultancy Services, Gujarat State Fertilizer Company and Gujarat Energy Research and Management Institute. Further, the institute has been declared as an Institute of National Importance by an Act of Parliament. The major objective of its establishment is to set up a model of education which can produce best-in-class human resources in IT and harnessing the multidimensional facets of IT in various domains.

Programme Objectives

After undergoing this course, the students will be able to design and develop programs for an agent to learn and act in a structured environment. The course has a very strong laboratory component based on Python and Matlab/Octave.


The laboratory sessions are designed to give participants with the hands-on experience of dealing with tabular methods to solve RL problems. The following will be included in the laboratory: Exploration-Exploitation in n-arm bandit, Sequential decision making, Robot path planning, G-Bike bicycle transfer problem, Elevator scheduling problem. Participants will have a better learning experience if they carry their laptops.

Course Contents

Review of Probability and Stochastic Processes: Conditional Probability, Bayes rule, Random variables, Random Process, Expectations and Conditional Expectation

Markov Decision Process: Introduction to random process, Markov Process, MDP Formulation, Utility theory, utility function, value iteration, policy iteration, partially observable MDPs

Reinforcement Learning: n-armed bandit, Finite Markov Decision Process, Dynamic Program, Monte Carlo Methods, Temporal Difference Learning, n-step Bootstrapping, Planning and Learning with Tabular Methods: Q-learning, Value-iteration, Policy-iteration, Approximation based methods

Workshop Chair

Prof. Sarat Kumar Patra


IIIT Vadodara


Prof. Suman Mitra


DAIICT, Gandhinagar

Dr. Pratik Shah

Assistant Professor

IIIT, Vadodara

Dr. Jignesh Bhatt

Assistant Professor

IIIT, Vadodara

Dr. Ashish Phophalia

Assistant Professor

IIIT, Vadodara

Dr. Bhupendra Kumar

Assistant Professor

IIIT, Vadodara