Machine Learning A

In Fall 2023, Machine Learning A (MLA) will be taught by Christian Igel,  Yevgeny Seldin, and Sadegh Talebi. The course covers theoretical and practical foundations of machine learning. The course assumes knowledge of linear algebra, calculus, probability theory, and basic programming skills. Please, check the following self-assessment assignment to test whether you have the necessary qualifications for the course. If you have difficulties solving it, check our preparing yourself for the course page.

We plan to cover the following topics:

• Basics of probability theory and the meaning of generalization beyond a sample under the i.i.d. assumption

• K-Nearest Neighbors algorithm

• Perceptron

• Linear Regression

• Logistic Regression

• Regularization and working in the feature space

• Markov's and Hoeffding's inequalities and their application to derivation of generalization bounds

• Generalization bounds based on training/validation/test sets

• Lower bound for generalization (the impossibility of generalization in the worst case)

• Occam's razor bound and its application to decision trees

• Random Forests

• Neural Networks

• Principal Component Analysis (PCA)

• K-means clustering

Practicalities

Programming Language: The working language of the course is Python. See the programming exercise in the self-assessment assignment to verify whether you are ready.

Course dates: The next round of the course will run from the 4th of September 2023 till the 3rd of November 2023 with a take-home written exam taking place in the last week of the course. We usually offer TA classes in the week prior to the course start to help with math and programming prerequisites.

Lectures: As of 2023, the course is placed in Schedule B. Specifically, the lectures will take place on Mondays 9:15-12:00 and Fridays 9:15-12:00. 

TA classes: We expect to have about 12 TA groups/classes and we allow our students to join any TA class they like and if they need they can attend more than one TA session. The TA sessions will be 3 hours long and will focus on going through solutions of assignments that have been submitted as well as help with ongoing course material. The TA classes will be spread throughout the week, so it should not be a problem to find one that fits your schedule. Exact details about the time of TA classes will be provided later. We guarantee TA classes on Tuesdays (afternoon) and Fridays (afternoon), but there will be some other available time slots.

Home assignments: There will be weekly home assignments including theoretical and practical questions. We expect to have 6-7 assignments in total. 

Final exam: The final exam is a 5-day take-home written exam, but it is designed in a way that it should be solvable within 2.5 days (assuming you have followed the material throughout the course), see the dates above. We provide 5 days to accommodate for potential overlap with exams in other courses and other commitments you might have (e.g., part-time job). The final exam includes theoretical and practical questions and must be solved individually.