Mathematics for Machine and Deep Learning

CPGEI

General Information

Objective: Study of the main mathematical tools for machine and deep learning methods.

Syllabus: Linear Algebra, Vector Calculus, Probability and Distributions, Optimization, Implementation of Classical ML models.

Duration/credits: 45 hours/3 credits (12 weeks).

Time: Fridays (08:20 - 12:00) - online with synchronous online (SYN) and assynchronous (ASYN) classes.

Grade: Assignments (100%).

Lecturers: André Eugenio Lazzaretti.

Bibliography and Supporting Materials

Book:

Deisenroth, M. P.; Faisal, A. A. & Ong, C. S. (2020), Mathematics for Machine Learning, Cambridge University Press.
Solution guide (link).

Week 1 - General overview

Content:

Slides (link).
Video (link).

Week 2 - Linear Algebra

Content:

Slides (link).
Videos:
- Part 1 (link).
- Part 2 (link).
- Part 3 (link). Erratum: include Ex. 2.20 as a final exercise!

Assignments:

- Proposed exercises: 2.6, 2.7, 2.8b, 2.3, 2.10, 2.12, 2.16c, 2.16d, 2.19, 2.20. Check the video to understand the suggested order.

Week 3 - Analytic Geometry

Content:

Slides (link).
Video:
- Part1 (link).
- Part 2 (link).

Assignments:

- Proposed exercises: 3.1, 3.3, 3.4, 3.5, 3.6, 3.8, 3.10. Check the video to understand the suggested order.

Week 4 - Matrix Decompositions

Content:

Slides (link).
Videos:
- Part 1 (link).
- Part 2 (link).

Assignments:

Proposed exercises: 4.1, 4.3, 4.4, 4.7, 4.8, 4.9. Check the video to understand the suggested order.

Week 5 - Vector Calculus

Content:

Slides (link).
Videos:
- Part 1 (link).
- Part 2 (link).

Assignments:

- Proposed exercises: 5.2, 5.3, 5.4, 5.5, 5.7, 5.8. Check the video to understand the suggested order.

Week 6 - Probability and Distributions

Content:

Slides (link).
Videos:
- Part 1 (link).
- Part 2 (link).
- Part 3 (link).
  - Due to the questions regarding Sufficient Statistics, this video may help to complement the proper definition and intepretation.
- Part 4 (Extra): Parameter Estimation with ML, MAP, and Bayesian approaches (link, tutorial).

Assignments:

Proposed exercises: 6.1, 6.4, 6.2, 6.5, 6.12. Check the video to understand the suggested order.

Week 7 - Optimization

Content:

Slides (link).
Video (link).

Assignments:

Proposed exercises: 7.1, 7.6, 7.7, 7.8, 7.9, 7.10. Check the video to understand the suggested order.

Week 8 - When Models Meet Data

Content:

Slides (link).
Video (link).

Week 9 - Linear Regression

Content:

Slides (link).
Videos
- Part 1 (link).
- Part 2 (link).

Assignments:

Tutorial (link).
Solution (link).

Week 10 - Dimensionality Reduction with Principal Component Analysis

Content:

Slides (link).
Videos
- Part 1 (link).
- Part 2 (link).

Assignments:

Tutorial (link) - only Part 1!
Solution (link).

Week 11 - Density Estimation with Gaussian Mixture Models

Content:

Slides (link).
Videos
- Part 1 (link).

Assignments:

Tutorial (link).
Solution (link).

Week 12 - Classification with Support Vector Machines

Content:

Slides (link).
Videos
- Part 1 (link).