Abstract: Interactive imitation learning has emerged as a practical approach for solving complex control tasks, including robotics and video games. By making interactive queries to a demonstration expert, a learning agent can collect targeted feedback to refine its policy, often achieving significantly higher sample efficiency than methods that rely solely on offline expert demonstrations.

In this talk, we advance the theoretical understanding of the value of interaction in imitation learning from a sample-efficiency perspective. In the realizable setting, we show that when expert annotation cost is measured on a per-state basis, interactive imitation learning provides provable cost savings over offline approaches. In the presence of model misspecification, we further demonstrate that interactive methods can achieve stronger competitiveness guarantees relative to the expert policy than purely offline methods. Finally, motivated by practical implementations of imitation learning, we show that combining offline datasets with interactive expert annotations can, in some cases, be substantially more cost-efficient than relying on either source alone. We also verify our theoretical findings with simulations on MuJoCo control tasks.

Joint work with Yichen Li. Based on papers: https://arxiv.org/pdf/2412.07057, https://arxiv.org/pdf/2312.16860

Bio: Chicheng Zhang is an Assistant Professor in the Department of Computer Science at the University of Arizona. He is mainly interested in theory and applications of interactive machine learning, such as active learning, reinforcement learning, and imitation learning. He regularly serves as a program committee member and publishes in conferences in machine learning and learning theory, such as ICML, COLT, NeurIPS, ICLR. He was a recipient of an outstanding paper runner-up award at ICML 2022 and NSF CAREER Award in 2025.