Machine Learning B

In block 4 of the spring semester of 2025, Machine Learning B (MLB) will be taught by Nirupam Gupta, Amartya Sanyal and Yevgeny Seldin. This course is a continuation of Machine Learning A course, and provides a deeper understanding of machine learning algorithms. It presents the theoretical foundations of machine learning and introduces advanced learning techniques. Tentative list of topics to be covered:

• Basics of Optimization Theory

  • Important properties of functions

  • Constrained optimization and Lagrange duality

  • Stochastic gradient descent (SGD)

  • Other optimization methods, e.g., interior-point methods

• Basics of Information Theory

  • Entropy

  • Relative entropy (the Kullback-Leibler divergence)

  • The method of types

  • kl inequality for concentration of measure

• Advanced techniques for analyzing generalization power of learning algorithms

  • Vapnik-Chervonenkis (VC) analysis

  • VC analysis of SVMs

  • VC lower bound

  • PAC-Bayesian analysis

  • PAC-Bayesian analysis of majority vote

  • Bernstein-type concentration inequalities, with applications to analysis of learning algorithms

• Kernel Methods

  • Kernels and RKHS

  • SVMs

• Ensemble classifiers and weighted majority vote

  • Boosting technique

  • AdaBoost

  • XGBoost

• Non-linear dimensionality reduction

  • Stochastic neighbor embedding

  • The t-SNE algorithm

• Bayesian inference

  • Basic concepts

  • Difference between Bayesian and frequentist views

Practicalities

 The course will be taught in English.  

Credits: 7.5 ECTS

Level: Full Degree Master 

Prerequisites:  Strong mathematical background and familiarity with the basics of machine learning (for example, the completion of "Machine Learning A (MLA)" course offered in the fall semester). Please check the "Recommended Academic Qualifications" box on the official webpage of this course, and evaluate your eligibility for the course by going over this self-assessment assignment and self-preparatory material. 

Programming Language: The working language of the course is Python. See the programming exercise in the self-assessment assignment to verify whether you are ready.

Course dates: The course will run from the 22nd of April 2025 till the 22nd of June 2025.

Lectures: The lectures will take place on Wednesdays 10 :15 - 12:00 and  13:15 - 15:00. The lectures will be held at University of Copenhagen (North Campus), but they will be streamed via Zoom and video recordings will be uploaded to the internal course page. This means that it is possible to take the course fully remotely. 

TA classes: The TA sessions will be held on Mondays 13:15 - 15:00.  Exact details about the time of TA classes will be provided later. To support remote participation, we will have stream the TA sessions over Zoom. TA sessions will not be recorded. 

Home assignments: There will be weekly home assignments. We expect to have 6-7 graded assignments in total. There will be no final exam in the course, and the final grade will be determined by the cumulative points scored in the assignments. 

Official course website: For more details, refer to the official course webpage. 

Registration

The course welcomes applications from students enrolled at other universities as well as people from the industry. But it is also open to anyone interested in foundations of machine learning. All elements of the course (including the assessment) can be followed fully remotely, hence the course can in principle be taken by anyone.

You can register for the course using one of the links below.  Please click on the link that applies to you. 

• Credit Students (for those enrolled at a Danish educational institute, registration open from Nov 15 - Dec 2, 2024) 

• Exchange Students (enrolled at a non-Danish educational institute with exchange agreement with University of Copenhagen)

• Guest Students (enrolled at a non-Danish educational institute without exchange agreement with University of Copenhagen): 

  1. EU Students (enrolled at an EU/EEA or Swiss educational institute) 

  2. Non-EU Students (enrolled at an educational institute outside EU/EEA or Switzerland)

• Continuing Education Applicants (for applicants from industry or individuals not enrolled at any educational institute, etc.)

While ordinary registration for the course ends on December 2, 2024, you can also register late for the course in Jan 15 - 28, 2025 and March 26 - April 16, 2025. All the information on course registration can be found on the official course website mentioned below.

Contact

In case of questions, please reach out to the course organizer, Nirupam Gupta (nigu@di.ku.dk).