Course in Probability and Information Theory
(or The Theory of the Theories of Everything)
Matteo Marsili, The Abdus Salam ICTP, Trieste
Instructions for students: lectures are listed below. They are based on pre-recorded video-lectures and lecture notes or book chapters (click on the underlined texts). Some supplementary material (reading, video lectures, etc) is occasionally given. Exercises are given in the lecture notes. Exercises are a test that you properly understood the content of the lecture, so they are not optional. Live sessions (either in person or online) will be scheduled to answer your questions on the course material and/or on the exercises.
Part I
Introduction and outline
Reading: Eugene Wigner on the unreasonable effectiveness of mathematics
Listen and watch: Nima Arkani-Hamed on the morality of fundamental physics,
Paradoxes in probability theory
How to learn more efficiently?
2. Classical probability
More on classical probability
Reading: Feller Chapter 4
3. Independence and random variables (lecture notes)
Independence and conditional probability
Random Variables
*Sampling and urns
4. Generating functions (lecture notes)
Generating functions
Counting with functions
Probability generating functions
More on balls and boxes
Cumulant generating functions
5 Random Walks, Branching processes and Markov chains (lecture notes and exercises for the 2nd part)
First returns and last visits (see Feller Chapter 3)
Random walks with drift (see Feller XI)
Part II
6. Typical behavior (lecture notes)
Introduction
Limits in probability
Borel-Cantelli lemmas
7. Laws of large numbers (lecture notes)
Laws of large numbers
The Asymptotic Equipartition Property (Cover&Thomas c.3)
*What should we expect from the expected value?
8. Limit theorems for sums and extremes (lecture notes)
Limit theorems for sums
*Stable distributions and the renormalisation group
*Building blocks of stochastic processes
Limit theorems for extremes
Discretionary lecture
*When are models predictable?
9. Information theory (lecture notes)
10. Large Deviation Theory (lecture notes)
On the Legendre transform (see also this)
Gartner-Ellis theorem
Large deviations for fat tailed distributions
11. Maximum entropy (lecture notes) and Statistical Mechanics (lecture notes)
12. Statistical inference