Machine Learning

In fall 2019 Machine Learning (ML) will be taught by Yevgeny Seldin and Christian Igel. The course covers theoretical and practical foundations of machine learning. The course assumes knowledge of Linear Algebra, Calculus, Probability Theory, and basic programming skills. Please, check the following self-assessment assignment to test whether you have the necessary qualifications for the course. If you have difficulties solving it, check our preparing yourself for the course page.

We plan to cover the following topics:

  • Basics of probability theory and the meaning of generalization beyond a sample under the i.i.d. assumption

  • K-Nearest Neighbors algorithm

  • Perceptron

  • Linear Regression

  • Logistic Regression

  • Markov's and Hoeffding's inequalities and their application to derivation of generalization bounds

  • Generalization bounds based on training/validation/test sets

  • Lower bound for generalization (the impossibility of generalization in the worst case)

  • Occam's razor bound and its application to decision trees

  • Kernels and basic kernel methods

  • Support Vector Machines (SVM)

  • Vapnik-Chervoneniks (VC) analysis and VC-dimension

  • VC generalization bound for SVMs

  • VC lower bound

  • Random Forests

  • Neural Networks

  • Principal Component Analysis (PCA)

  • K-means clustering

Practicalities

Programming Language: The working language of the course is Python. See the programming exercise in the self-assessment assignment to verify whether you are ready.

Course dates: The course will run from 18 November 2019 till 15 January 2020 with a take-home written exam from 20 till 24 January (see below). We may offer TA classes in the week prior to the course start (11-15 November) to help with math and programming prerequisites - this is still to be decided on.

Lectures: The current (still tentative) plan is to have lectures on Wednesdays 9:15-12:00 and 13:15-16:00. The first lecture of the course will be held on Monday, 13:15-16:00, in the first week of the course.

TA classes: We expect to have about 12 TA groups/classes and we allow our students to join any TA class they like and if they need they can attend more than one TA session. The TA sessions will be 3 hours long and will focus on going through solutions of assignments that have been submitted + help with ongoing course material. The TA classes will be spread throughout the week, so it should not be a problem to find one that fits your schedule. Exact details about the time of TA classes will be provided later. We guarantee TA classes on Mondays 13:15-16:00, but there will be many more available time slots.

Home assignments: There will be weekly home assignments including theoretical and practical questions. We expect to have 6-7 assignments in total.

Final exam: The final exam is a 5-day take-home written exam, but it is designed in a way that it should be solvable within 2.5 days (assuming you have followed the material throughout the course), see the dates above. We provide 5 days to accommodate for potential overlap with exams in other courses and other commitments you might have (e.g., part-time job). The final exam includes theoretical and practical questions and must be solved individually.