We aim to provide a self-contained introduction of the mathematical and statistical foundations of machine learning at the (intermediate / advanced) undergraduate level. The core subject matter is a survey of the mathematical and statistical machinery underlying the main computational and programming tools used in practice, leading up to neural network-based machine learning. This course will involve some illustration using software, but it is not meant to provide a full hands-on introduction of the relevant software tools. Also, the course provides the foundation for but does not discuss neural network-based machine learning at length; deep learning should / could be the subject of a follow up course.

COURSE INFO

MATH 493B , Spring 2023, WVU (CRN: 18007, 3 hours)

Tu / Th 4:00 - 5:15, 403 Armstrong Hall

Instructor: Adam Halasz, PhD (Mathematics) -- email: halasz@math.wvu.edu

Target audience

The intended audience of this course includes engineering students, science majors in quantitative fields and advanced undergraduate Mathematics / Applied Mathematics students interested in exposure to and understanding how key machine learning methods work.

Machine learning in practice involves a relatively large set of mathematical (and statistical) concepts. However, it relies fundamentally on software tools and the practice of creating application specific workflows similar to protocols used in laboratory work. The innovation and know-how are centered on the usage of the various methods / tools, rather than the mathematical subtleties of any specific element in the workflows. Thus, for a practitioner it is important to be familiar with the basic features of a mathematical concept or method, but the derivation, deeper understanding, or associated paper-and-pencil skills, are less crucial. Therefore, the pace of this course will be quicker than that of a typical junior / senior Mathematics course.

ASSIGNMENTS & GRADING

There won't be any traditional in-class exams. We will focus on students' ability to work with the mathematical concepts rather than memorize definitions or prove theoretical results. Homework will be given at 1-2 week intervals; for the mathematical foundation parts of the course we will rely on problems from or similar to those in the respective text. Homework for the application part (final 4 weeks) as well as the two Projects will involve some programming. There will be two Projects in lieu of a midterm and final exam.

Homework: 6 - 8 assignments (50%) ; Midterm project (20%) ; Final project (30%)

Level / Expected Background

The primary text Mathematics for Machine Learning by Deisenroth, Faisal, and Ong [MML] is intended as an intermediate level textbook for undergraduate students interested in Machine Learning. We plan to offer it as a high level (senior) undergraduate course at WVU, aimed at students who have some background in general Mathematics. There are sections covering basic Linear Algebra, Vector Calculus, as well as Probabilities and Distributions; however, most of these should serve as reviews of the concepts, for students who have encountered them previously. Depending on the format and intended audience, we will complement this with other references as well as lecture notes discussing specific subjects or applications (pls. see below). The material will also be supplemented with chapters from Dive into Deep Learning by A. Zhang et al. [DDL]; this will also be the source of applications.

We see the ideal candidate for this course as an advanced undergraduate student with an interest in Machine Learning, typically an Applied Mathematics, Science, or Engineering major, who has successfully taken courses in at least one or two of: Univariate Calculus (Math 155 / 156), Linear Algebra (one of: Math 243, 343, 441), and basic Statistics (Stat 215). Alternatively, this course may also serve as a gateway to Data Science studies for more advanced (even graduate) Mathematics, Physical Science or Engineering students with a solid theoretical background, interested in boosting their career prospects outside academia.

Software / Programming Expectations

Machine learning in practice involves software and computational tools. This is not a programming course, but some basic experience with Python or another scientific programming language such as MATLAB, R, etc. is necessary. We plan to use examples in Python and will provide a tutorial. Like with other pre-requisites, a student with no prior exposure to programming whatsoever could in principle succeed but will find the course very difficult.

textBOOK[s]:

[MML] Mathematics for Machine Learning, by Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong; Cambridge University Press 2020. (link)

[DDL] Dive into Deep Learning, by A. Zhang et al. (link)

Additional references: Depending on the format and intended audience, we will occasionally complement this with [LLD] as well as lecture notes discussing specific subjects or applications:

[LLD] Linear Algebra and Learning from Data, by Gilbert Strang; Wellesley-Cambridge Press 2019. Website