Course materials
Week I
Video lecture on the definition of stochastic processes
Part1, Part2 lecture note 1 lecture note 2 slides computation
Assignment 1 (no need to be submitted. Solve it yourself)
Week II
Video lecture on the solution of the assignments pdf of the solution
Video lecture concerning the definition of the Markov chain Part 1 Part 2
Lecture notes on the examples of the Markov chain and some exercises (solve yourself)
Week III
Tutorial on January 18, 2021 Ballot theorem and reflection principle
Examples of Markov chains: Reading assignment for the Friday class (Section 2 is not included in the syllabus of the course and so read it for your own interest.)
The notes contain many problems. Please solve them yourself. We shall not discuss them in the tutorial class. Some similar problems will be discussed. If you get stuck or feel difficulty, please send me your notes on the problem over email. I will be happy to help you and share my comments. Help is only available if you send me your notes or work on the problem. No key to the solution will be provided.
Week IV
Tutorial on January 25, 2021 Study of the return times of SSRW Part I
Video lecture on the local behavior of the random walk near the origin
Class notes (Please see from page 14 to page 20)
Video lecture on Probability that the random walk stays above the x-axis for the first 2n time
Class notes (please see from page 21)
Video lecture on the probability distribution of the first and the last return time to the origin by SSRW
class notes (please see from page 26)
Video lecture on the probability distribution of the sojourn time and the maximum
Class notes (please see from page 32)
The journey of a SSRW (handwritten notes which cover all the above topics)
Week V
Notes from the tutorial class on February 01.
Video lecture on the decomposition of the state space
Lecture notes Proofs of the theorems
Weak VI
Quiz (question paper) on February 8.
Assignment 2 (Due date 16th February). The class is divided into 10 groups. Each group is assigned to solve problems and will present their solution on 18th February and each group will have 10 minutes to present their solution. The details of the group members and problems assigned to them will be found here.
February 11. We shall continue our discussion on the decomposition of the state space of the Markov chain. The concept of recurrence and transience will be introduced. Here is the notes from the class. As the recorder broken, I could not provide you the recording. The following is the link for the video lecture (recorded privately) and it contains a little bit more than I covered in the class.
Video lecture on the classification of the state space Notes from the lecture
Lecture notes on the classification of state space
Week VII
Video Lecture on Recurrence and Transience Part II Notes used in the video lecture Lecture notes Proofs of the theorems
Some exercises from HOEL PORT AND STONE which are relevant to the syllabus of the midsem. We shall discuss some of them in the class on 15th February.
The solutions to the problems in the Assignment can be found here (we shall update it if necessary)
Week IX
Lecture notes for the live class on March 08. We proved that recurrence and transience is class property. We started discussing on the birth-death chain.
Notes on birth-death chain (please read it from Hoel Port Stone).
Dissection of Markov chain part I
Lecture notes Video lecture Lecture notes used in the video lecture Handwritten general proof of the theorem
Week X
Quiz (Question paper)
Offline discussion on Birth-Death chain (recurrence and transience + derivation P_z(\tau_x < \tau_y) + introduction to the instability property of branching chain)
Online discussion on Birth-Death chain (notes of the lecture)
Dissection of Markov chain part II
Lecture notes Video lecture Lecture notes used in the video lecture Handwritten general proof of the theorem
Week XI
Online interaction on Birth-Death chain ( derivation P_z(\tau_x < \tau_y) + introduction to instability of Branching chain)
Offline interaction (Instability of Branching chain)
Week XII
Absorption probabilities (Markov chain on finite state space)
Lecture notes video lecture notes used in the video lecture
Video lecture II notes used in the lecture
Online interaction on March 29
Online interaction on April 01
Assignment III (To be submitted by April 16.). In case of any doubts or confusion or typo, please contact me as soon as possible.
Here is the problem allocation to the groups. Checked solutions are available here
Absorption probabilities (Markov chain on countably infinite state space)
Lecture notes (notes may contain some error or step jumps might be there. Use it in your own risk. Reference is HPS.)
Stationary distribution of a Markov chain
Additional Materials
Ergodic theorem for Markov chain
Lecture notes Video lecture I Video lecture II
Consequence of Ergodic theorem to find out the stationary distribution
Lecture notes (Reading assignment)
Weak convergence of the Markov chain
Continuous-time Markov chain: Poisson process
Lecture notes Video lecture