Probability and Statistics I
(summer 2022)

Time and Place

  • Lecture: Mondays 9:00 to 10:30 in S4

  • Tutorial: Thursdays 14:00 to 15:30 in S11 given by Gaurav Sunil Kucheriya (see link to website below)

Office hours:

By email appointment.

Topics

Probability

  1. Axioms of probability, basic examples (discrete and continuous). Conditional probability, the law of total probability, Bayes' theorem.

  2. Random discrete variables: expectation, variance, linearity of expectation and its use. Basic discrete distributions.

  3. Continuous random variables: description using probability density function. Basic continuous distributions.

  4. Independent random variables. Random vectors (marginal distribution). Covariance, correlation.

  5. Laws of large numbers, basic inequalities (Markov, Chebyshev, Chernoff), Central limit theorem.

Statistics

  1. Point estimates: unbiased estimates, confidence intervals.

  2. Hypothesis testing, significance level. Two-sample tests.

  3. Test of goodness of fit, test of independence.

  4. Non-parametric estimates.

  5. Bayesian and Frequentist Approach. Maximum a posteriori method, Least mean square estimate.

  6. Maximum-likelihood method. Bootstrap resampling.

  7. Simulation, generation of random variables from a distribution. Monte Carlo simulation.

  8. Informative: Markov chains.

Lecture content

14.2: polynomial identity testing, probability space, uniformly at random, discrete and continuous prob space, Bernoulli cube, conditional prob, independence

21.2: mutual vs pairwise independence, whp & almost surely, union bound, isolated vertices of random graphs, law of total probability, gambler's ruin (setup)

28.2: gambler's ruin (calculations), continuity of probability, Bayes theorem, covid testing, random variables, probability mass function (PMF), cumulative distribution function (CDF), Bernoulli distribution, Binomial distribution

7.3: Poisson distribution, Geometric distribution, expected value, LOTUS, linearity of expectation, variance, conditional expectation, law of total expectation

14.3: expectation and variance of various distributions, random vectors, marginal distributions, independence, multinomial distribution, coupling, LOTUS, expectation of product, linearity of expectation, conditional distribution

21.3: splitting the Poisson distribution, continuous random variables, probability density function (PDF), Uniform distribution, Exponential distribution, standard Normal distribution

28.3: general Normal distribution, Gamma distribution, covariance and correlation, correlation vs causation, probability integral transform

4.4: inverse transform sampling, Markov inequality, converting randomized algorithms, Chebyshev inequality, weak law of large numbers, central limit theorem, random sample, statistic, estimator

11.4: empirical CDF, sample mean, sample variance, bias of estimators, mean square error, point estimates, method of moments, maximum likelihood estimator

25.4: interval estimates, confidence interval, Student's t-distribution, hypothesis testing, null hypothesis, type I/II error, significance level, p-value, single sample test

2.5: two sample test, goodness of fit, chi squared distribution, linear regression

9.5: Simpson's paradox, p-hacking, non-parametric statistics, permutation test, bootstrapping

16.5: rejection sampling, Bayesian statistics, maximum a-posteriori, least mean square, Markov chains

Lecture notes

You can download the lecture notes and the R code segments using the password shared via email.

Literature

  1. Bartoszynski and Niewiadomska-Budaj: Probability and Statistical Inference, J. Wiley, 1996.

  2. Mitzenmacher and Upfal: Probability and Computing, Cambridge, 2005.

  3. Bertsekas and Tsitsiklis: Introduction to Probability, Athena Scientific, 2008.

  4. The Book of Statistical Proofs

Links