Machine Learning Reading Group at UArizona

Welcome to our Machine Learning Reading Group (MLRG) at the University of Arizona, organized by Kwang-Sung Jun, Jason Pacheco, and Chicheng Zhang. We have a weekly meeting starting from Fall 2019. The goal is to pick a specific topic to focus each semester, read papers together, and hopefully apply them to your research, collaborate, and write papers together. By the end of the semester, we at least understand the fundamental concepts and challenges of the topic, what people have proposed so far, and what open problems are out there!

The current topic

Fall 2019: Imitation Learning

To get started on imitation learning, we can read Hal Daume III's book chapter, and MDP basics (e.g. Sutton and Barto, Chapter 3). This ICML workshop has many useful resources: https://sites.google.com/view/icml2018-imitation-learning/

Here is a list of papers on imitation learning that Chicheng collected; you are welcome to find more papers of your interest (both theory and applications) and send them to us.

Interactive imitation learning:

  • Hal Daumé III, John Langford, Daniel Marcu. Search-based Structured Prediction. Machine Learning Journal 2009.
  • Stephane Ross, Geoffrey J. Gordon, J. Andrew Bagnell. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS 2011.
  • Stephane Ross, J. Andrew Bagnell. Reinforcement and Imitation Learning via Interactive No-Regret Learning. NIPS 2014.
  • Wen Sun, Arun Venkatraman, Geoffrey J. Gordon, Byron Boots, J. Andrew Bagnell. Deeply AggreVaTeD: differentiable imitation learning for sequential prediction. ICML 2017.
  • Ching-An Cheng and Byron Boots. Convergence of Value Aggregation for Imitation Learning. NIPS 2017.
  • Wen Sun, Anirudh Vemula, Byron Boots, J. Andrew Bagnell. Provably Efficient Imitation Learning from Observation Alone. ICML 2019.

Inverse reinforcement learning:

  • Andrew Y. Ng and Stuart Russell. Algorithms for Inverse Reinforcement Learning. ICML 2000.
  • Brian D. Ziebart, Andrew Maas, J.Andrew Bagnell, Anind K. Dey. Maximum Entropy Inverse Reinforcement Learning. AAAI 2008.
  • Brian D. Ziebart, J.Andrew Bagnell, Anind K. Dey. Modeling Interaction via the Principle of Maximum Causal Entropy. ICML 2010.
  • Jonathan Ho, Stefano Ermon. Generative Adversarial Imitation Learning. NIPS 2016.
  • Kareem Amin, Nan Jiang, Satinder Singh. Repeated Inverse Reinforcement Learning. NIPS 2017.

Apprenticeship learning:

  • Pieter Abbeel and Andrew Y. Ng. Apprenticeship Learning via Inverse Reinforcement Learning. ICML 2004.
  • Umar Syed and Robert E. Schapire. A Game-Theoretic Approach to Apprenticeship Learning. NIPS 2007.
  • Umar Syed, Michael H. Bowling, Robert E. Schapire. Apprenticeship Learning Using Linear Programming. ICML 2008.
  • Alekh Agarwal, Ashwinkumar Badanidiyuru, Miroslav Dudik, Robert Schapire, Aleksandrs Slivkins, Miro Dudík. Robust Multi-objective Learning with Mentor Feedback. COLT 2014.

Schedule

10/25/2019

  • The first meeting.
  • Possible topics discussed: (1) Monte Carlo tree search (2) imitation learning (3) sequential information maximization (4) Bayesian sparse structural learning (5) Bayesian optimization (6) Bayesian deep learning
  • We eventually chose imitation learning by voting.
  • homework: read the tutorial by Daume III

11/01/2019

11/08/2019