Invited Speakers


We have the following invited speakers:


Title: The NetHack Learning Environment


Abstract:

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both these things. In this talk, I present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based rogue like game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open-source at https://github.com/facebookresearch/nle .


Bio:

Tim is a Research Scientist at Facebook AI Research (FAIR) London and a Lecturer at the Centre for Artificial Intelligence in the Department of Computer Science at University College London (UCL). Prior to that, he was a Postdoctoral Researcher in Reinforcement Learning at the University of Oxford, a Junior Research Fellow in Computer Science at Jesus College, and a Stipendiary Lecturer in Computer Science at Hertford College. Tim obtained his Ph.D. at UCL, and was awarded a Microsoft Research Ph.D. Scholarship in 2013 and a Google Ph.D. Fellowship in 2017. His work focuses on developing agents that are intrinsically motivated to explore complex open world environments, and that learn to acquire and utilize commonsense, world and domain knowledge to systematically generalize to novel situations.


Title: Open-ended environments for advancing RL


Abstract:

The field of reinforcement learning is pushed forwards by the presence of challenging environments. Over the years, the complexity of these environments has continued to increase, but the question is how can we continue to push the complexity of environments with respect to the optimal policy complexity in a scalable manner. Here I will discuss using multi-agent environments to create more open-ended environments, and discuss examples of our work to move in this direction with Capture the Flag and Starcraft 2. Finally I will discuss some future directions for generating even more open-ended environments to further push our RL algorithms.


Bio:

Max Jaderberg is a research scientist at DeepMind, working on the intersection of deep learning, reinforcement learning, and multi-agent systems. His recent work includes creating the first agent to beat human professionals at StarCraft II published in Nature, and creating algorithms for training teams of agents to play with humans in Capture the Flag published in Science. He previously co-founded Vision Factory, a computer vision startup, which was acquired by Google in 2014, and completed a PhD at the Visual Geometry Group, University of Oxford.



Title: Collaborative Construction and Communication in Minecraft


Abstract:

I will present work done by my group on defining a collaborative construction task that allows us to use the Minecraft platform to study situated natural language generation and understanding.

In this task, one player (the Architect) needs to instruct another (the Builder) via a chat interface to construct a given target structure that only the Architect is shown.

I will discuss what makes this task interesting and challenging. I will also describe models that we have developed for the Architect and the Builder role, and discuss what remains to be done to create agents that can solve this task.


Bio:

Julia Hockenmaier is an Associate Professor and Willett Faculty Scholar in Computer Science at the University of Illinois. She is a recipient of the NSF CAREER award and of the IJCAI-JAIR best paper award. She was a program chair for EMNLP 2018, and is a past chair of the board of the North American Chapter of the Association for Computational Linguistics (NAACL).


Title: Collaboration in Situated Instruction Following


Abstract:

I will focus on the problem of executing natural language instructions in a collaborative environment. I will propose the task of learning to follow sequences of instructions in a collaborative scenario, where two agents, a leader and a follower, execute actions in the environment and the leader controls the follower using natural language. To study this problem, we build CerealBar, a multi-player 3D game where a leader instructs a follower, and both act in the environment together to accomplish complex goals. I will focus on learning an autonomous follower that executes the instructions of a human leader. I will briefly describe a model to address this problem, and a learning method that relies on static recorded human-human interactions, while still learning to recover from cascading errors between instructions.


Bio:

Yoav Artzi is an Assistant Professor in the Department of Computer Science and Cornell Tech at Cornell University. His research focuses on learning expressive models for natural language understanding, most recently in situated interactive scenarios. He received an NSF CAREER award, paper awards in EMNLP 2015, ACL 2017, and NAACL 2018, a Google Focused Research Award, and faculty awards from Google, Facebook, and Workday. Yoav holds a B.Sc. summa cum laude from Tel Aviv University and a Ph.D. from the University of Washington.


Title: What an Agent Knows: Evaluation in Open Worlds


Abstract:

Agents tackling complex problems in open environments often benefit from the ability to construct knowledge. Learning to independently solve sub-tasks and form models of the world can help agents progress in solving challenging problems. In this talk, we draw attention to challenges that arise when evaluating an agent’s knowledge, specifically focusing on methods that express an agent’s knowledge as predictions. Using the General Value Function framework we highlight the distinction between useful knowledge and strict measures of accuracy. Having identified challenges in assessing an agent’s knowledge, we propose a possible evaluation approach that is compatible with large and open worlds.


Bio:

Alex Kearney is a PhD student studying Computer Science at the University of Alberta. She is supervised by Rich Sutton and Patrick Pilarski. She focuses on Artificial Intelligence and Epistemology.

Her research addresses how artificial intelligence systems can construct knowledge by deciding both what to learn and how to learn, independent of designer instruction. She predominantly uses Reinforcement Learning methods.


Title: Endless Frontiers?


Abstract:

The research community is gradually coming to a realization that policies trained arcade-like video games are very limited. They overfit badly and are not going to take us far along the way to some sort of general intelligence. This should perhaps not be surprising, given that such games generally have tightly defined tasks, fixed perspectives, and generally static worlds. More and more attention is therefore given to games that are in some sense open-ended or feature open worlds. Could such games be the solution to our problems, allowing the development of more general artificial intelligence? Perhaps, but basing competitions or benchmarks on open-ended games is not going to be easy, as the very features which make for a good benchmark are the same that lead to brittle policies. Shoe-horning open-world games into a standard RL framework is unlikely to be the best option for going forward. Many of the most interesting opportunities for developing intelligent behavior is likely to come from agents constructing their own challenges and environments. The boundary between playing a game and constructing a world is not well-defined: I will give examples from where the same RL setup was used to play SimCity and to develop game levels. I will also briefly introduce the Generative Design in Minecraft Competition, which focuses on building believable settlements.


Bio:

Julian Togelius is an Associate Professor in the Department of Computer Science and Engineering, New York University, USA. He works on artificial intelligence for games and on games for artificial intelligence. His current main research directions involve procedural content generation in games, general video game playing, player modeling, and fair and relevant benchmarking of AI through game-based competitions. Additionally, he works on topics in evolutionary computation, quality-diversity algorithms, and reinforcement learning. He is the Editor-in-Chief of the IEEE Transactions on Games. Togelius holds a BA from Lund University, an MSc from the University of Sussex, and a PhD from the University of Essex. He has previously worked at IDSIA in Lugano and at the IT University of Copenhagen.