Theory of Probability lecture notes
(Exercises are included at the end of each section)
1. Probability Spaces. Probability spaces, some examples. Dynkin's theorem, regularity of measures, Ulam's theorem.
2. Random Variables. Random variables, independence, expectation.
3. Kolmogorov’s Consistency Theorem. Kolmogorov's theorem about the existence of stochastic processes.
Laws of Large Numbers
4. Inequalities for Sums of Independent Random Variables. Chebyshev, Hoeffding, Hoeffding-Chernoff, Bennett, Bernstein inequalities. Azuma's inequality and chromatic number of random graph.
5. Laws of Large Numbers. Borel-Cantelli lemma, weak and strong laws of large numbers.
6. 0 – 1 Laws. Random Series. Kolmogorov and Hewitt-Savage 0-1 laws. Convergence of random series. Kolmogorov's strong law of large numbers.
7. Stopping Times, Wald’s Identity, Markov Property. Plus another proof of the strong law of large numbers.
Central Limit Theorem
8. Convergence of Laws. Convergence of laws, Selection theorem.
9. Characteristic Functions. Smoothness, Uniqueness, stability of normal distribution, Fourier inversion, Levy's continuity theorem.
10. The Central Limit Theorem. Lindeberg's approach, method of characteristic functions, multivariate normal distributions.
11. Lindeberg’s CLT. Three Series Theorem. Plus an example of Poisson approximation.
Conditional distributions and martingales
12. Conditional Expectations and Distributions. Conditional expectations. Conditional distributions, existence, product space case, disintegration.
13. Martingales. Uniform Integrability. Examples of martingales, Doob's decomposition, uniform integrability of right-closable (sub)martingales.
14. Stopping Times. Properties of stopping times, Optional Stopping theorem, fundamental Wald's identity.
15. Doob’s Inequalities and Convergence of Martingales. Doob's inequalities, convergence of (sub)martingales.
Metrics on probability measures
16. Bounded Lipschitz Functions. Properties of bounded Lipschitz functions.
17. Convergence of Laws on Metric Spaces. Portmanteau theorem, metrics for weak convergence, convergence of empirical measures, uniform tightness, Prokhorov's theorem.
18. Strassen’s Theorem. Relationship between Ky Fan and Levy-Prokhorov metrics: modeling close laws by random variables close in probability.
19. Kantorovich-Rubinstein Theorem. Wasserstein metrics and Kantorovich-Rubinstein duality theorems.
20. Brunn-Minkowski and Prekopa-Leindler Inequalities. Applications: Wasserstein distance and Kullback-Leibler divergence; Gaussian concentration.
21. Stochastic Processes. Brownian Motion. Definition, existence of Brownian motion and Brownian bridge.
22. Donsker Invariance Principle. Example of convergence to Brownian motion.
23. Convergence of Empirical Process to Brownian Bridge. Symmetrization, Kolmogorov’s chaining and Dudley's entropy bound.
24. Reflection Principles for Brownian Motion. Strong Markov property, and various distributions related to Brownian motion and Brownian bridge.
25. Skorohod’s Imbedding and Laws of the Iterated Logarithm.
Representations of exchangeable arrays
26. Moment problem and de Finetti's theorem for coin flips. Moment problem, Hausdorff's theorem, de Finetti's theorem for coin flips.
27. The general de Finetti and Aldous-Hoover representation. General case of de Finetti's theorem, Aldous-Hoover represenation for two-dimensional arrays.
28. The Dovbysh-Sudakov representation. Dovbysh-Sudakov representation for Gram-de Finetti arrays.
29. Poisson processes. Poisson processes: mapping, superposition, marking, equivalence, existence.