Dr. Robert Loftin, Delft University of Technology (TU Delft)

Abstract:

Learning to cooperate with other agents (human or AI) will be essential for intelligent systems to succeed in many real-world applications. The challenge is that while we (or the agents we design) are trying to learn to cooperate with other agents, those agents may simultaneously be learning to cooperate with us. Practical and theoretical approaches to learning in cooperative settings therefore assume that other agents' behaviors are stationary, or else make very specific assumptions about other agents' learning processes. The goal of our work is to understand whether we can reliably learn to cooperate with others without such restrictive assumptions, which are unlikely to hold in real-world settings. In this talk, I will discuss a set of recent impossibility results which show that, in fact, no learning algorithm can be guaranteed to learn to cooperate with all possible adaptive partners. This is true even when every possible partner is guaranteed to eventually cooperate with some fixed strategy that our learner could have committed to. This talk will conclude with a discussion of the practical implications of these results for human-AI cooperation, and potential theoretical solutions to the issues they raise.