The Schedule

With the deep learning breaching into all fields of science and engineering, the advancement in deep learning field is also going at a faster pace. The algorithms and methods have evolved with influx of techniques from optimization, mathematics, visualization and data engineering. This talk is designed to provide a comprehensive coverage of machine learning topics related to deep learning and its applications.

Markov chains play an important role in understanding time correlation of processes. We will briefly introduce the discrete time Markov chains and some of its properties, and discuss some of the conditions under which stationary distribution exists. We will conclude with an application for the Markov chains.

This talk will introduce the basic multi-armed bandit: a sequential optimization problem that forces an agent to balance exploration and exploitation. Bandits are simple special cases of reinforcement learning problems. We will look at common algorithmic techniques to solve bandit learning problems -- the epsilon-Greedy and Upper Confidence Bound (UCB) strategies -- and characterize their performance. We will also discuss extensions to bandit problems beyond the vanilla version.

This talk will be an overview of stochastic approximation algorithms. Beginning with a somewhat detailed overview of the so called `ODE' (for `Ordinary Differential Equations') for their analysis, I shall describe several variants such as multiple timescales, distributed and asynchronous variants, etc., followed by key applications such as stochastic gradient descent, fixed point computations, etc.

We will look at reversible Markov chains, the example of random walks on graphs, and will then study Markov Chain Monte Carlo (MCMC) sampling methods for random sampling from high-dimensional probability distributions. Independent sampling from such distributions is often difficult. Instead, MCMC gives samples from a Markov chain whose equilibrium distribution is the target distribution. We will discuss aspects of the MCMC method related to the starting point, the rate of convergence to equilibrium, the burn-in phase, and some methods of sampling.

In this talk, we will mainly discuss the policy evaluation problem in reinforcement learning and look at methods based on Temporal Difference (TD) learning to solve the same. In policy evaluation, we are given a policy, i.e., a strategy, to perform a task involving sequential decisions and the goal is to evaluate its quality in terms of its value function. We will see that a simple gradient based approach to evaluate this value function leads to the Monte-carlo method. This method, unfortunately, is offline in nature. A genius trick to transform this into an online algorithm was proposed by Richard Sutton, thereby paving the way for the development of the TD methods. If time permits, we will discuss the convergence analysis of the TD(0) method, an important member of the TD family. In the latter half of the talk, we shall once again look at the problem of policy evaluation, but this time focusing on the situation where the state space is enormous, possibly infinite. In such cases, it is more realistic to find an approximation to the actual solution than the actual solution itself. With this in mind, we shall discuss the idea of function approximation. We shall end with a discussion on convergence of the TD(0) method with function approximation and compare and contrast the properties of the limit with that of the original TD(0) method.

This talk will be in three parts. In the first part, we start with applications of game theory and present several essential definitions and examples in game theory. In the second part, we define the central notion of Nash equilibrium and cover foundational results in non-cooperative game theory. In the third part, we introduce central notions in cooperative game theory.

Mechanism design is the design of games with desirable behaviour. We first provide motivational examples for mechanism design from different application domains. We then present important desirable properties of mechanisms and provide an overview of key results in mechanism design with concrete examples from auctions and market design.