An Introduction to Probability Theory
Finally, I thought of putting the online lectures which were taken by me for the M.Sc. in Engineering Mathematics in ICT Mumbai during the lockdown. To have a good hold on data science, it is essential to have a good understanding of probability distributions. As per the feedback from the students, the quality of the recordings is good. They have also suggested uploading the lectures so that others can access them. The course structure of the M.Sc. in Engineering Mathematics in ICT, Mumbai is available on the official website of the Department of Mathematics. The following lectures are tailored to give adequate exposure to the students (primarily B.Sc. in Mathematics) in their first semester so that they can follow Statistics and Machine Learning courses in later semesters. In the latter semesters, they study Statistical Inference, Statistical Computing, Machine Learning, Deep Learning and AI, and Stochastic Process. All lectures are shared based on students' requests. Anybody should be able to access it and no need to worry about acknowledgement. If you find it useful, that is enough. If anybody facing difficulty in accessing the recordings, just drop an email to ar.bhowmick@ictmumbai.edu.in or amiyaiitb@gmail.com.
Lecture I
1) Experiment, Sample Space, and Algebra of Events
2) Sequence of Events and Limiting events
3) Sigma Algebra and its examples
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture II
1) Axiomatic Definition of Probability Function
2) Calculus of Probabilities
3) Examples
4) Continuity of Probability Function
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture III
1) Conditional Probability and Independence
2) Conditional Probability as a reduction of the sample space
3) Theorem of Total Probability
4) Bayes theorem
5) Some discussion problems related to conditional probability and Bayes theorem
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture IV
1) Discussion on limsup and liminf of a sequence of events
2) Borel-Cantelli lemma with proof.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture V
1) Cumulative Distribution Function
2) Distribution of Random variable
3) Basic calculation
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture VI
1) Sigma algebra generated by a random variable
2) How to show that a function is a random variable
3) Problems with the random variable
4) Sequence of functions of random variables and related problems.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture VII
1) Distribution of Random variables
2) Properties of Distribution Function
3) Probability Mass function and Probability Density function (examples)
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture VIII
1) Distribution function of transformation of Random variable
2) Examples of discrete and continuous random variables.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture IX
1) Probability Integral Transform
2) Expected Value of Random variables and function of random variables
3) Variance of Random Variables
4) Examples of some common distributions
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture X
1) Central moments and Raw moments and their computation
2) Existence of Moments
3) Moment generating function
4) The moment problem
5) Connection between moments and the tail probabilities
R Codes used in today's lecture:
# X binomial(n,p)
par(mfrow=c(1,1))
n = 40
p = 0.05
fun_binom = function(x){
choose(n, x)*p^x*(1-p)^(n-x)
}
x = 0:n
prob_binom = fun_binom(x)
print(prob_binom)
plot(x, prob_binom, type = "h", lwd = 2, col = "red", xlim = c(0, 10))
points(x, prob_binom, cex = 1.5, pch = 19, col = "red")
lambda = n*p
fun_Poisson = function(x){
exp(-lambda) * lambda^x /factorial(x)
}
x = 0:n
prob_Poisson = fun_Poisson(x)
print(prob_Poisson)
lines(x+0.2, prob_Poisson, type = "h", col = "magenta", lwd = 2)
points(x+0.2, prob_Poisson, cex = 1.5, pch = 19, col = "magenta")
legend("topright", c("Binomial", "Poisson"), col = c("red", "magenta"), lwd = c(2,2), bty = "n")
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XI
1) Some common distribution
2) Normal distribution and MGF
3) Memoryless property of exponential distribution
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XII
1) Problem discussion
2) Probability Inequalities
3) Identities (Stien's lemman and applications)
4) Location-Scale Family with examples
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XIII
1) Joint probability Mass function
2) Joint Probability Density Function
3) Joint Distribution Function
4) Expectation of Functions of random variables
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XIV
1) Conditional Distribution and Independence
2) Conditional Expectation and Conditional Variance
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XV
1) Bivariate transformation
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XVI
1) Bivariate transformation continued. Examples discussed.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XVII
1) Multivariate distributions (examples)
2) Problem discussion.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XVIII
1) Visualization of some sampling distribution. This has been done in the lab lectures.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XIX
In this lecture, we have talked about convergence concepts: Convergence in probability and convergence in distribution and how to visualize these ideas using R. Demonstration of Central Limit Theorem. Markov Inequality and its proof.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XX
1) Proof of Central Limit Theorem
2) Slutsky's theorem
3) Demonstration of Central Limit Theorem
4) Almost Sure convergence and its example.
Link to the Video: (Part A) (Part B)
Link to the Writing Notes: Click Here
Lecture XXI
1) Approximation of Expectation of function of random variables using first-order and second-order Taylor expansion.
2) Approximation of the variance
3) Generalization of the Central Limit Theorem
You must do the exercises given and should be implemented using R.
Link to the Video: Click Here
Link to the Writing Notes: Click Here
Lecture XXII
1) Sampling from Normal distribution
2) Independence of Sample mean and sample variance
3) Chisquare distribution
4) t distribution
Link to the Video: Click Here
Link to the Writing Notes: Click Here