Lecture 1: Overview, functions of a single variable
Lecture 2: Linear algebra review (vectors, matrices, eigenstuff, diagonalization)
Bonus: A (musical) review of when matrices are invertible
Lecture 3: Derivative tests in R^n
Lecture 4: Positive/Negative (Semi)Definite Matrices
Lecture 5: Sylvester's Criterion & classifying critical points
Lecture 6: Closed & bounded sets, coercive functions, extreme value theorem
Lecture 7: Convex sets & convex combinations
Lecture 8: Convex functions
Lecture 9: "Building" convex functions, methods for showing that a function is convex
Lecture 10: More "building" convex functions, concrete examples, Jensen's inequality
Lecture 11: The Arithmetic Mean - Geometric Mean (A-G) Inequality, examples
Lecture 12: Geometric Programming
Lecture 13: General comments on optimization duality, examples of "badly behaved" geometric programs
Lecture 14: Polynomial interpolation & lines of best fit
Lecture 15: Least squares fit
Lecture 16: Orthogonality, Gram-Schmidt
Lecture 17: Minimum-norm problems
Lecture 18: Generalized inner products and minimum-H-norm problems
Lecture 19: Obtuse angle criterion
Lecture 20: Separation theorem, Bolzano-Weierstrass
Concrete example of the separation theorem
Lecture 21: Extreme value theorem, support theorem, subgradients of convex functions
Lecture 22: General non-linear programs, perturbations of convex programs
Lecture 23: Sensitivity vector lemma, "goalposts" lemma, saddle-point version of KKT theorem
Lecture 24: Gradient form of KKT, concrete example
Lecture 25: KKT duality
Lecture 26: Constrained geometric programming
Lecture 27: The general GP dual
Lecture 28: Penalty method, absolute value penalty function, Courant-Beltrami penalty function
Lecture 29: Convergence to optimal solutions in the penalty method
Lecture 30: Coercive functions and the penalty method
Lecture 31: The penalty method & KKT duality
Lecture 32: Equality constraints
Lecture 33: Equality constraints and geometric programs, in-class review
Lecture 34: Newton's Method
Lecture 35: Newton's Method in R^n
Lecture 36: Method of Steepest Descent
Lecture 37: Beyond Steepest Descent
Lecture 38: Choosing step size and descent direction
Lecture 39: Broyden's Method
Lecture 40: Sherman-Morrison formula, end-of-semester logistics
Practice problems
As mentioned in the syllabus, I highly encourage typesetting assignments rather than writing by hand. You may wish to do so using Overleaf, which is free and cloud-based. If you would find it helpful, I have created a basic template for homework in this course (the TeX code is available here). You are not required to use this template or to use LaTeX.
Mathematics:
Linear Algebra Refresher, by Thomas R. Shemanske
A complete linear algebra course available via MIT OpenCourseWare, by Gilbert Strang
Writing:
"Guidelines for good mathematical writing," by Francis Su.
Detexify, a web app for identifying the LaTeX command(s) for various symbols.
Overleaf, a cloud-based LaTeX editor.