Course materials

Week I


  1. Video lecture on the definition of stochastic processes

Part1, Part2 lecture note 1 lecture note 2 slides computation

  1. Exercise on conditional probability

  2. Assignment 1 (no need to be submitted. Solve it yourself)

Week II

  1. Video lecture on the solution of the assignments pdf of the solution

  2. lecture notes in the tutorial class

  3. Video lecture concerning the definition of the Markov chain Part 1 Part 2

  4. Notes used in the video lecture part 1

  5. Lecture notes on the examples of the Markov chain and some exercises (solve yourself)

Week III

  1. Tutorial on January 18, 2021 Ballot theorem and reflection principle

Class notes Handwritten notes

  1. Examples of Markov chains: Reading assignment for the Friday class (Section 2 is not included in the syllabus of the course and so read it for your own interest.)

The notes contain many problems. Please solve them yourself. We shall not discuss them in the tutorial class. Some similar problems will be discussed. If you get stuck or feel difficulty, please send me your notes on the problem over email. I will be happy to help you and share my comments. Help is only available if you send me your notes or work on the problem. No key to the solution will be provided.

Week IV

  1. Tutorial on January 25, 2021 Study of the return times of SSRW Part I

Class notes

  1. Video lecture on the local behavior of the random walk near the origin

Class notes (Please see from page 14 to page 20)

  1. Video lecture on Probability that the random walk stays above the x-axis for the first 2n time

Class notes (please see from page 21)

  1. Video lecture on the probability distribution of the first and the last return time to the origin by SSRW

class notes (please see from page 26)

  1. Video lecture on the probability distribution of the sojourn time and the maximum

Class notes (please see from page 32)


The journey of a SSRW (handwritten notes which cover all the above topics)

Week V

  1. Notes from the tutorial class on February 01.

  2. Video lecture on the decomposition of the state space

Lecture notes Proofs of the theorems


Weak VI

  1. Quiz (question paper) on February 8.

  2. Assignment 2 (Due date 16th February). The class is divided into 10 groups. Each group is assigned to solve problems and will present their solution on 18th February and each group will have 10 minutes to present their solution. The details of the group members and problems assigned to them will be found here.

  3. February 11. We shall continue our discussion on the decomposition of the state space of the Markov chain. The concept of recurrence and transience will be introduced. Here is the notes from the class. As the recorder broken, I could not provide you the recording. The following is the link for the video lecture (recorded privately) and it contains a little bit more than I covered in the class.

Video lecture on the classification of the state space Notes from the lecture

Lecture notes on the classification of state space


Week VII

  1. Video Lecture on Recurrence and Transience Part II Notes used in the video lecture Lecture notes Proofs of the theorems

  2. Some exercises from HOEL PORT AND STONE which are relevant to the syllabus of the midsem. We shall discuss some of them in the class on 15th February.

  3. Class notes on 15th February

  4. The solutions to the problems in the Assignment can be found here (we shall update it if necessary)


Week IX

  1. Lecture notes for the live class on March 08. We proved that recurrence and transience is class property. We started discussing on the birth-death chain.

  2. Notes on birth-death chain (please read it from Hoel Port Stone).

  3. Dissection of Markov chain part I

Lecture notes Video lecture Lecture notes used in the video lecture Handwritten general proof of the theorem


Week X

  1. Quiz (Question paper)

  2. Offline discussion on Birth-Death chain (recurrence and transience + derivation P_z(\tau_x < \tau_y) + introduction to the instability property of branching chain)

  3. Online discussion on Birth-Death chain (notes of the lecture)

  4. Dissection of Markov chain part II

Lecture notes Video lecture Lecture notes used in the video lecture Handwritten general proof of the theorem

Week XI


  1. Online interaction on Birth-Death chain ( derivation P_z(\tau_x < \tau_y) + introduction to instability of Branching chain)

  2. Online interaction on March 25

  3. Offline interaction (Instability of Branching chain)

Week XII


  1. Absorption probabilities (Markov chain on finite state space)

Lecture notes video lecture notes used in the video lecture

Video lecture II notes used in the lecture


Online interaction on March 29


Online interaction on April 01


Assignment III (To be submitted by April 16.). In case of any doubts or confusion or typo, please contact me as soon as possible.

Here is the problem allocation to the groups. Checked solutions are available here


Online interaction on April 5

Absorption probabilities (Markov chain on countably infinite state space)

Lecture notes (notes may contain some error or step jumps might be there. Use it in your own risk. Reference is HPS.)

  1. Stationary distribution of a Markov chain

Lecture notes


Additional Materials

Ergodic theorem for Markov chain

Lecture notes Video lecture I Video lecture II

Consequence of Ergodic theorem to find out the stationary distribution

Lecture notes (Reading assignment)

Weak convergence of the Markov chain

Lecture notes Video lecture

Continuous-time Markov chain: Poisson process

Lecture notes Video lecture