Discrete Optimization and Machine Learning

Ksenia Bestuzheva (SoSe 2024 / Seminar)

Both Machine Learning methods as well as Discrete Optimization methods are important tools in many real-world applications. In this course we will primarily study the interplay of integer programming and, more broadly, discrete optimization methods and machine learning.
The format of the course is strongly research-oriented requiring active participation of the students. The course will be a mix of student presentation, discussions, and project work. In the first few weeks in-class projects will be defined.

Course organization

Prerequisites: Linear Algebra, Analysis, and Discrete Optimization (ADM I/ADM II)

Registration: Together with paper selection after first meeting (seminar outline)

Participation requirements:
Students are expected to individually

Write a report (5 page minimum in Latex) about the paper and the contribution from the students themselves.
Send your reports to Jannis Halbey
During the semester, present their plans for the final report and presentation in a shorter 5-minute presentation.
Give a final presentation to present their findings (details discussed in the first meeting).

Paper/project selection:
Students choose one of the papers below to work on

Up to 2 students can work on the same paper
Assignment is first come, first served
Send paper choice to Jannis Halbey
For fairness reasons we only accept selections after 19.04.2024 04:00 pm

Contribution:
Every student is expected to make a contribution to the given topic and not just reproduce the results. The aim is not to obtain groundbreaking new results, but to get an impression of scientific work. Furthermore, you also need to have a very sound understanding of the subject in order to contribute something yourself. Contributions can be an extension of the original algorithm, a new theoretical proof or a comparison with new research.

Examples for strong contributions:

COIL: A Deep Architecture for Column Generation

- Identify the OptNet-Layer as one main bottleneck
- Examine Lagrangian Dual Framework from Ferdinando Fioretto et al.: Lagrangian Duality for Constrained Deep Learning as alternative solution
- Implement modification and evaluate empirically

Sparse Adversarial and Interpretable Attack Framework

- Identify vanilla Frank-Wolfe Algorithm as potential point for improvement
- Examine the variant Blended Pairwise Conditional Gradient
- Implement modification and evaluate empirically

PEP: Parameter Ensembles by Perturbation

- Comparison with state-of-the-art Deep Ensembles
- Examine advantages and disadvantages of both methods
- Implement combination of both approaches and evaluate empirically

Timeline:
All meetings take place in the MAR building in Marchstraße 23.