Abstract

Learning with multiple objectives emerges frequently as a new unified learning paradigm from recent machine learning problems such as learning under fairness and safety constraints; learning across multiple tasks including multi-task learning and meta-learning; learning across multiple agents including federated and multi-agent learning; and, learning with hierarchical games including incentive designs, reward shaping, and Stackelberg game. This tutorial will introduce principled frameworks that cover bilevel optimization for learning under two objectives with preference, and multi-objective optimization for learning with multiple objectives without preference, and their combinations. Efficient algorithms will be presented and recent advances on optimization and generalization theory will be introduced. We will highlight how we can apply those algorithms and theories to multi-task learning and learning with Markov games. Upon completion, the audience is expected to gain the necessary knowledge to effectively perform learning tasks with multiple objectives in and beyond the aforementioned applications.

2:00 - 2:20 PM - Introduction and Background (T. Chen)

• New challenges of learning under multiple objectives

• Two optimization toolboxes to address those challenges

• History of bilevel and multi-objective optimization

2:20 - 3:20 PM - Bilevel Optimization Foundations (T. Chen)

• Solution concepts and metrics of optimality

• Implicit gradient-based bilevel methods

• Optimality condition-based bilevel methods

• Value function-based penalty bilevel methods

3:20 - 3:30 PM - Bilevel Applications to Reinforcement Learning (Z. Yang)

• Introduction to bilevel reinforcement learning settings 

3:30 - 4:00 PM - Coffee break 

4:00 - 4:50 PM - Applications to Reinforcement Learning (Z. Yang)

• Examples of bilevel reinforcement learning

• Optimization aspects of bilevel reinforcement learning 

• Statistical aspects of bilevel reinforcement learning 

4:50 - 5:50 PM - Multi-objective Optimization Foundations (L. Chen)

• Preferences based on partial ordering

• Solution concepts and measures of optimality

• Multi-gradient methods (MGDA, CAGRAD, SMG, MoCo, MoDo)

• epsilon-constraint methods

• Theory in multi-objective learning

• Applications to compute vision and speech processing

5:50 - 6:00 PM - Conclusions and Open Directions