Tentative list of topics

• Review of probability theory

• Review of linear algebra

• Vector spaces, linear independence, linear mappings

• Norms, inner products, positive definiteness, orthogonality, projections, and rotations

• Matrix decompositions & vector calculus

• Determinants, eigenanalysis, EVD and SVD

• Gradients of vector-valued functions and matrices

• Principal Component Analysis

• Numerical optimization

  • Constrained vs. unconstrained optimization, optimality conditions

• Convex sets and functions

• Unconstrained optimization algorithms

  • Gradient descent

  • Accelerated gradient descent

  • Newton’s method, gradient projection

  • Stochastic gradient descent (SGD)

• Machine learning motivations, basic formulation, mini-batch SGD

  • Linear classification

• Constrained optimization

• Bayesian inference

  • Approximate inference

• Additional topics

  • Generative processes, and generative modeling

  • TBD