July 23rd, 2021

Workshop on

Human-AI Collaboration in Sequential Decision-Making

at ICML'21


A key challenge for the successful deployment of many real world human-facing automated sequential decision-making systems is the need for human-AI collaboration. Effective collaboration ensures that the complementary abilities and skills of the human-users and the AI system are leveraged to maximize utility. This is for instance important in applications such as autonomous driving, in which a human user’s skill might be required in safety critical situations, or virtual personal assistants, in which a human user can perform real-world physical interactions which the AI system cannot. Facilitating such collaboration requires cooperation, coordination, and communication, e.g., in the form of accountability, teaching interactions, provision of feedback, etc. Without effective human-AI collaboration, the utility of automated sequential decision-making systems can be severely limited. Thus there is a surge of interest in better facilitating human-AI collaboration in academia and industry. Most existing research has focused only on basic approaches for human-AI collaboration with little focus on long-term interactions and the breadth needed for next-generation applications. In this workshop we bring together researchers to advance this important topic.

Invited Speakers

Andreea Bobu
UC Berkeley

Scott Niekum
UoT at Austin

Katja Hofmann
Microsoft Research

Dorsa Sadigh
Stanford University

Sarah Sebo
University of Chicago

Yisong Yue


For the schedule of the workshop please visit https://icml.cc/virtual/2021/workshop/8367.

Accepted Papers

We received many high-quality submissions to the workshop and are happy to announce that we were able to accept the following papers for presentation at the workshop:

  • Kolby Nottingham, Anand Balakrishnan, Jyotirmoy Deshmukh and David Wingate, "Using Logical Specifications of Objectives in Multi-Objective Reinforcement Learning"

  • Ishaan Shah, David Halpern, Michael Littman and Kavosh Asadi, "Convergence of a Human-in-the-Loop Policy-Gradient Algorithm With Eligibility Trace Under Reward, Policy, and Advantage Feedback"

  • Vahid Balazadeh Meresht, Abir De, Adish Singla and Manuel Gomez Rodriguez, "Learning to Switch Among Agents in a Team"

  • H M Sajjad Hossain, Yash Chandak, Soundararajan Srinivasan, David Koleczek, Weihao Tan, Siddhant Pradhan, Vishal Rohra, Vivek Chettiar, Aaslesha Rajaram, Nicholas Perello and Nan Ma, "Intervention Aware Shared Autonomy"

  • Hengyuan Hu, Adam Lerer, Brandon Cui, David Wu, Luis Pineda, Noam Brown and Jakob Foerster, "Off Belief Learning"

  • Brandon Cui, Hengyuan Hu, Luis Pineda and Jakob Foerster, "K-level Reasoning for Zero-Shot Coordination in Hanabi"

  • Hamsa Bastani, Osbert Bastani and Wichinpong Sinchaisri, "Improving Human Decision-Making with Machine Learning"

  • Jeevana Inala, Yecheng Ma, Osbert Bastani, Xin Zhang and Armando Solar-Lezama, "Safe Human-Interactive Control via Shielding"

  • Jennifer Suriadinata, William Macke, Reuth Mirsky and Peter Stone, "Reasoning about Human Behavior in Ad Hoc Teamwork"

  • Gottipati Vijaya Sai Krishna, Cloderic Mars, Greg Szriftgiser, Sagar Kurandwad, Francois Chabot and Vincent Robert, "Cogment: Open Source Framework For Distributed Multi-actor Training, Deployment And Operations"

  • Minori Narita, Sandhya Saisubramanian, Roderic A. Grupen and Shlomo Zilberstein, "Identifying Missing Features in State Representation for Safe Decision-Making"

  • Hengyuan Hu, Samuel Sokota, David Wu, Anton Bakhtin, Andrei Lupu, Noam Brown and Jakob Foerster, "Self-Explaining Deviations for Zero-Shot Coordination"

  • Aakriti Kumar, Trisha Patel, Aaron Benjamin and Mark Steyvers, "Explaining Algorithm Aversion with Metacognitive Bandits"

  • Daniel Shin and Daniel Brown, "Offline Preference-Based Apprenticeship Learning"

  • Wenshuo Guo, Kumar Agarwal, Aditya Grover, Vidya Muthukumar and Ashwin Pananjady, "Learning from an Exploring Demonstrator: Optimal Reward Estimation for Bandits"

Call for Papers

We welcome submissions to the workshop under the general theme of ''Human-AI Collaboration in Sequential Decision-Making". Topics of interest include, but are not limited to:

  • Models for accountability and trust in sequential decision making

  • Learning adaptive behavior from and with non-stationary agents

  • Robust multi-agent learning under partial observability and perception/reward mismatch

  • Applications, use-cases, and studies of human-AI collaboration, for example in domains such as computer games, robotics, programming, data analytics, etc.

  • Interpretability of decision making policies (both from human’s & AI’s perspective)

  • Learning about intent and goals from demonstrations in collaborative setting

  • Leveraging complementary abilities and skills for safety critical applications

Submission Format

We welcome original submissions as well as submissions that have been recently published, or accepted for publication in a conference or journal.

Submissions should be up to 4-8 pages excluding references, acknowledgements, and supplementary material, and should follow the ICML format. Submission will be via CMT. The review process will be double-blind to avoid potential conflicts of interests.

There will be no official proceedings, but the accepted papers will be made available on the workshop website. Accepted papers will be presented as a spotlight talk and poster.

Please submit papers at https://easychair.org/conferences/?conf=humanaiicml21

Presentation at the workshop

Accepted submissions will be presented as posters; a subset of all submissions will further be selected to be presented as spotlight talks (~5 minutes).

Important Dates

  • Submission deadline: June 3, 2021 (AOE)

  • Acceptance notification: June 14, 2021 June 10, 2021

  • Final papers + spotlight videos due: June 30, 2021 June 20, 2021


Please refer to the ICML website for registration details as they become available.

The Organizers

Besmira Nushi

Microsoft Research

Adish Singla

Max Planck Institute for Software System

Sebastian Tschiatschek

Unversity of Vienna


If you have any questions or comments please don't hesitate to contact us at human.ai.collab.icml21ws@gmail.com.