Romuald Elie (DeepMind)

Romuald Elie is research Scientist at DeepMind and associate professor of applied mathematics at University Gustave Eiffel. His research topics lie at the intersection of dynamic programming, optimal decision making, machine learning and game theory. His main contributions are the construction of efficient Monte Carlo methods for the resolution of complex planning problems, the design of optimal incentives rewards in imperfect information games and the study of learning mechanisms in mean field games. He has hold previous positions at ETH Zurich, University Paris Dauphine, University of Michigan and University of California SB & Berkeley.

Talk title: Mean Field Learning with Infinitely Many Agents

Talk abstract: Learning by experience in Multi-Agent Systems (MAS) is a difficult and exciting task, due to the lack of stationarity of the environment, whose dynamics evolves as the population learns. This talk focuses on the design of efficient and scalable algorithms for systems with a large population of identical interacting agents (e.g., swarms). For this purpose, a promising approach relies on focusing on the asymptotic limit case where the size of the population is infinite, leading to the consideration of now well established so-called mean field games. In particular, the optimal policy derived on asymptotic mean field games provides an approximate Nash equilibrium in the finite number of players setting. After presenting an overview of the intuitions and main properties driving the theory of mean field games, we will present our contributions to a burgeoning research field: model free learning of Nash equilibrium in Mean field Games through repeated experience.

Thore Graepel (DeepMind)

Thore Graepel is a research group lead at Google DeepMind and holds a part-time position as Chair of Machine Learning at University College London. He studied physics at the University of Hamburg, Imperial College London, and Technical University of Berlin, where he also obtained his PhD in machine learning in 2001. After postdoctoral work at ETH Zurich and Royal Holloway College, University of London, Thore joined Microsoft Research in Cambridge in 2003. At DeepMind since 2015, Thore leads the multi-agent research team and contributed to AlphaGo, the first computer program to defeat a human professional player in the full-sized game of Go.

Talk title: From AlphaGo to MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Talk abstract: Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess and Go, where a perfect simulator is available. However, in real-world problems the dynamics governing the environment are often complex and unknown. In this work we present the MuZero algorithm which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. MuZero learns a model that, when applied iteratively, predicts the quantities most directly relevant to planning: the reward, the action-selection policy, and the value function. When evaluated on 57 different Atari games - the canonical video game environment for testing AI techniques, in which model-based planning approaches have historically struggled - our new algorithm achieved a new state of the art. When evaluated on Go, chess and shogi, without any knowledge of the game rules, MuZero matched the superhuman performance of the AlphaZero algorithm that was supplied with the game rules.

Edward Lockhart (DeepMind)

Edward is a research engineer at DeepMind where he has worked on a variety of research topics, including efficient neural speech synthesis, relational representation learning, and reinforcement learning in two-player games. Prior to DeepMind, Edward had a 20-year career in finance, leading quantitative research teams in equities, foreign exchange, and interest rates. He has a BA in Mathematics from the University of Cambridge, and is an expert (although infrequent) bridge player.

Talk title: OpenSpiel: A Framework For Reinforcement Learning in Games

Georgios Piliouras (SUTD)

Georgios Piliouras is an assistant professor at the Singapore University of Technology and Design (SUTD). He received his PhD in Computer Science from Cornell University in 2010. He has held postdoc positions at the Georgia Institute of Technology (GaTech, ECE Dept.) and California Institute of Technology (Caltech, Dept. of Computing and Mathematical Sciences). He has held visiting positions at UC Berkeley and DeepMind. He is the recipient of a Singapore NRF Fellowship (2018) and a Simons/UC Berkeley Fellowship (2015).

Talk title: Beyond equilibrium: A dynamical systems approach to learning in games

Abstract: Historically, Nash equilibrium has been the predominant solution concept in games. We review a recent stream of results on multi-agent learning in standard classes of games (both competitive, e.g. zero-sum games as well as cooperative, e.g. potential games) that showcase how the behavior of standard learning dynamics can deviate in unexpected ways from the predictions of equilibrium play. A wide range of behaviors is possible, and in fact common, such as cycles, bifurcations, chaos and even simultaneous local stability of Nash equilibrium and chaos. We will end by discussing open questions and opportunities emerging for multi-agent reinforcement learning.