Welcome to Ke Sun's Homepage.

Insightful Talks (continually updated)

- Control Perspective for RL (John Tsitsiklis (MIT)): The Shades of Reinforcement Learning
- Anytime-valid Inference via Betting (Ian Waudby-Smith (CMU)): Estimating means of bounded random variables by betting
- Double Robustness (Stefan Wager (Stanford)): Double Robustness for Average Treatment Effect
- Physical Intelligence vs Language Intelligence (Pulkit Agrawal (MIT)): Why it's harder for AI to open doors than play chess?
- A New Paradigm of RL/Agent Research (Shunyu Yao, OpenAI, 2025): The Second Half
- The Future of RL Research (Richard Sutton, University of Alberta, 2025): Welcome to the Era of Experience

Learning Materials

Modern Statistics
- Foundations:
  1. Probability and Measure with videos by Adam B Kashlak
  2. High-dimensional Probability with video by Roman Vershynin
  3. High-dimensional Statistics by Martin J. Wainwright
  4. Asymptotics Statistics by Aad W Van der Vaart
- Advanced:
  1. Robust and Non-parametric Statistics (course 1 and course 2)
  2. Empirical Process: introduction, applications to semi-parametric model, and Weak Convergence and Empirical Processes
  3. Causal Inference:
    - - A First Course in Causal Inference by Peng Ding
      - Causal Inference: A Statistical Learning Approach by Stefan Wager
      - Applied Causal Inference from Nathan Kallus
  4. Experiment Design (lectures)
  5. Testing Statistical Hypothesis
  6. Nonlinear Time Series by Jianqing Fan and Qiwei Yao
  7. Information Theory
  8. Optimal Transport course
  9. Bayesian Optimization by Roman Garnett
Machine Learning:
- ML Theory: ML Theory Lecture from Csaba Szepesvári
- Statistical ML:
  1. Statistical Learning Methods (Chinese) by Hang Li
  2. The Elements of Statistical Learning (ESL) by Trevor Hastie, Robert Tibshirani and Jerome Friedman
  3. Statistical ML (course 1 & course 2 by Ryan Tibshirani and Larry Wasserman, course 3 by Danica Sutherland)
- Probabilistic ML: Probabilistic Machine Learning: Advanced Topics by Kevin Patrick Murphy
- Classical Textbooks: (1) Understanding Machine Learning: From Theory to Algorithms. (2) Foundations of Machine Learning (3) Computer Age Statistical Inference.
Deep Learning:
- Introduction: Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
- Theory: Mathematical Analysis of Machine Learning Algorithms by Tong Zhang
Reinforcement Learning:
- Introduction:
  1. Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto [the first book to learn RL]
  2. Introduction to RL (Coursera) by Matha White and Adam White
  3. Deep RL courses by Sergey Levine
  4. Reinforcement Learning by Emma Brunskill (Stanford CS234)
  5. Introduction to RL by Lucas Janson (Harvard)
  6. Reinforcement Learning: An Overview by Kevin M. Murphy [modern progress in different directions of RL]
- Theoretical RL:
  1. CMPUT 653 RL Theory by Csaba Szepesvári
  2. Theory of RL by Ambuj Tewari
  3. Foundations of RL (PKU Mooc) by Zhihua Zhang
- Control Theory:
  1. Stochastic Control and Decision Theory (McGill)
- Statistical RL:
  1. Statistical RL courses by Nan Jiang
  2. Reinforcement Learning: Theory and Algorithms by Alekh Agarwal Nan Jiang Sham M. Kakade Wen Sun

My Study Notes

Machine Learning: 1) Learning theory and Deep Double Decent 2) Neural Tangent Kernel 3) Concentration inequality 4) ODE and PDE-based Neural Networks 5) Distance, Optimal Transport and Wasserstein GANs 6) Unsupervised learning: Self-supervised learning, and Generative models 7) Adversarial machine learning 8) RKHS and Kernel Method.
Statistics (Fundamental Statistics Ideas): 1) Exploration Data Analysis, e.g, Dimension Reduction: t-SNE, 2) Overparameterized models and regularization 3) Adaptive Decision making. 4) Causal inference 5) Robustness 6) Bayes inference framework 7) Generic algorithms, e.g., Variational Inference, VAE and MCMC 8) Bootstrapping and simulation-based inference. Other: Statistical Inference, Influence Function.
Optimization (theory and algorithms): 1) Continuity, Smoothness and Convexity 2)Sampling, Zero-order and Langevin dynamics 3) GD, SGD and Proximal 4) Second-order and its approximation techniques 5) Minimax and bi-level Optimization 6) Tricks: initialization, BatchNormalization, Soft label learning and knowledge distillation