Course in Probability and Information Theory
(or The Theory of the Theories of Everything)
Matteo Marsili, The Abdus Salam ICTP, Trieste
(or The Theory of the Theories of Everything)
Matteo Marsili, The Abdus Salam ICTP, Trieste
Instructions for students: lectures are listed below. They are based on pre-recorded video-lectures and lecture notes or book chapters (click on the underlined texts). Some supplementary material (reading, video lectures, etc) is occasionally given. Exercises are given in the lecture notes. Exercises are a test that you properly understood the content of the lecture, so they are not optional. Live sessions (either in person or online) will be scheduled to answer your questions on the course material and/or on the exercises.
Introduction and outline (up to chapter 2)
Reading: Eugene Wigner on the unreasonable effectiveness of mathematics
Listen and watch: Nima Arkani-Hamed on the morality of fundamental physics,
                Paradoxes in probability theory
Overview of the course and introduction
What is probability?
*Subjective probability and lotteries
*Probability as an extension of logic to plausible reasoning
2. Classical probability (chapter 3)
More on classical probability
Reading: Feller Chapter 4
3. Independence and random variables (chapter 4 and 5)
Independence and conditional probability
Random Variables
*Sampling and urns
4. Generating functions (chapter 7)
Generating functions
Counting with functions
Probability generating functions
More on balls and boxes
Cumulant generating functions
5 Random Walks, Branching processes and Markov chains (chapters 9 to 11)
First returns and last visits (see Feller Chapter 3)
Random walks with drift (see Feller XI)
6. Typical behavior (chapter 13)
Introduction
Limits in probability
Borel-Cantelli lemmas
7. Laws of large numbers (chapter 14)
Laws of large numbers
The Asymptotic Equipartition Property (Cover&Thomas c.3)
*What should we expect from the expected value?
8. Limit theorems for sums and extremes (chapter 15)
Limit theorems for sums
*Stable distributions and the renormalisation group
*Building blocks of stochastic processes
Limit theorems for extremes
*When are models predictable?
9. Information theory (chapter 16)
10. Large Deviation Theory (chapter 17)
Missing Video-lecture on large deviations (finite support)
Missing Video-lecture on large deviations (thin tails)
On the Legendre transform (see also this)
Gartner-Ellis theorem
Large deviations for fat tailed distributions
11. Maximum entropy (chapter 18) and Statistical Mechanics (chapter 19)
12. Statistical inference (chapter 20)
Exercises for the second part are in chapter 21