How to Work With Real Humans in Human-AI Systems
Times
At NeurIPS: December 11, AM
At AAMAS: May 30, AM
Organizers
Elizabeth Bondi-Kelly (University of Michigan), ecbk@umich.edu
Krishnamurthy (Dj) Dvijotham (Google)
Matthew E. Taylor (University of Alberta & AI Redefined)
Panelists
Diana Acosta-Navas (Loyola University Chicago)
Susanne Gaube (University College London)
Hussein Mozannar (Massachusetts Institute of Technology)
Brief Description
As more and more AI systems are deployed in the real world, it becomes imperative to study these systems with real humans to avoid unexpected negative consequences during deployment. Yet, this can be challenging for researchers with more experience designing algorithms and less experience running human participant experiments, or deploying systems in the real world. In this tutorial, we will discuss the state of the human-AI collaboration field, emphasizing (i) incorporating humans into AI systems, including multi-agent, machine learning, and reinforcement learning systems, (ii) investigating when to rely on human vs. AI strengths, and (iii) designing human-AI studies to evaluate algorithms with real humans.
Expected Gained Skills
Discuss human-AI interaction paradigms, e.g., deferral, decision aids, active learning, etc.
Describe several cognitive biases present in human-AI decision-making.
Propose and discuss designs for evaluating human-AI systems.
Seek interdisciplinary collaborations to further study human-AI systems.