Probability and Statistics

Syllabus

Unit 1 : Sample space, probability axioms, real random variables (discrete and continuous), cumulative distribution function, probability mass/density functions, mathematical expectation, moments, moment generating function, characteristic function, discrete distributions: uniform, binomial, Poisson, geometric, negative binomial, continuous distributions: uniform, normal, exponential. 15L 

Unit 2 : Joint cumulative distribution function and its properties, joint probability density functions, marginal and conditional distributions, expectation of function of two random variables, conditional expectations, independent random variables, bivariate normal distribution, correlation coefficient, joint moment generating function (jmgf) and calculation of covariance (from jmgf), linear regression for two variables. 15L 

Unit 3 : Chebyshev’s inequality, statement and interpretation of (weak) law of large numbers and strong law of large numbers. Central Limit theorem for independent and identically distributed random variables with finite variance, Markov Chains, Chapman-Kolmogorov equations, classification of states. 10L 

Unit 4 : Random Samples, Sampling Distributions, Estimation of parameters, Testing of hypothesis. 20L 

Some questions related to Probability

Joint Distributions and Independence

Characteristic Functions

Random Vectors

Markov and Chebyshev Inequalities

Chernoff Bounds

Jensen's Inequality

Law of Large Numbers

The law of large numbers has a very central role in probability and statistics. It states that if you repeat an experiment independently a large number of times and average the result, what you obtain should be close to the expected value. There are two main versions of the law of large numbers. They are called the weak and strong laws of the large numbers. The difference between them is mostly theoretical. In this section, we state and prove the weak law of large numbers (WLLN). 

Sequence of Random Variables

Different Types of Convergence for Sequences of Random Variables

Convergence in Distribution

Convergence in Probability

Convergence in Mean

Almost Sure Convergence