AAAI 2023 Tutorial

Bi-level Optimization in Machine Learning:

Foundations and Applications

AAAI 2023 Tutorial

February 07, 2023 (2:00 pm – 6:00 pm EDT)

Room 206

Overview

Bi-level optimization (BLO) involves solving two coupled optimization tasks in a hierarchical manner. It has introduced broad interest and exhibited significant modeling power across a wide spectrum of contemporary applications in artificial intelligence (AI), such as robust AI, efficient AI, generalizable AI, and automated AI. This tutorial will systematically review various aspects of BLO and provide a holistic understanding of BLO from both theoretical and application views. This tutorial will include an in-depth analysis of the interaction between the theory and applications to fully realize the potential of the BLO paradigm. In particular, we will offer:

a concise overview of the important mathematical characterizations of the BLO problem
a precise understanding of BLO-oriented problem structures and algorithmic characteristics demanded in diverse applications and peering into theoretical convergence results of BLO
insights to why BLO formulation, theories, and algorithms for AI applications can yield significant performance boosts, as compared with the traditional, and more heuristic algorithms
a set of comprehensive experimental evaluations for BLO algorithms and applications
a code walkthrough for hands-on experience on the BLO algorithms, applications, and most importantly practical implementation tricks