• 09:00 - 09:10    (10 minutes) - Opening Remarks

• 09:10 - 09:25 (15 minutes)  - Invited Talk: Prof. Erin Talvitie (Harvey Mudd) - 20 Years of Asking the Wrong Questions in MBRL.

Abstract: Model-based reinforcement learning (MBRL), i.e. the problem of learning a predictive model and using it to plan, seems like a straightforward and intuitively sensible algorithmic proposition. Doing it well could transform the way we approach many core problems in RL. And yet, making sustained, robust progress toward this goal has proved surprisingly, infuriatingly difficult. We'll take a high-level tour of some garden paths and blind alleys that have caused useful shifts in my own frame, ending on the hopeful note that maybe, just maybe, some of the right questions are starting to come into view.

• 09:25 - 09:40 (15 minutes) - Invited Talk: Prof. George Konidaris (Brown) - Agents Must Learn Their Own Frames.

Abstract: I will argue that: 1) the right outer frame for a generally-intelligent agent is a decision process, 2) that that decision process must necessarily be drastically overpowered, compared to the natural framing of any individual task it may wish to solve, and that therefore 3) a critical component for achieving general intelligence is the ability to autonomously construct task-specific frames, which can be achieved by learning mutually-compatible observation and action abstractions. Finally, I will briefly discuss how 4) a reinforcement-learning agent able to do so can access the rest of AI.

• 09:40 - 09:55 (15 minutes) - Invited Talk: Prof. Mark Ho (NYU) - Why Might Having a Mind Be Useful?

Abstract: Reinforcement learning studies systems that generate reward-maximizing behavior. Where does the idea of a mind fit into this picture, if it’s even needed? Should we be behaviorists, cognitivists, or something in between? I will discuss some of the reasons why RL might want to take minds seriously, despite some of the challenges involved in studying them.

• 09:55 - 10:10 (15 minutes) - Invited Talk: Dr. Clare Lyle (Google DeepMind) - Beyond Optimality: Designing Open-Ended RL Agents.

Abstract: Reinforcement learning has proven to be an outrageously effective framework in settings where a) reward is well-defined and b) it is possible to quickly collect a diverse set of experiential data containing a gradient of reward signals. However, we have yet to see RL agents propose novel paradigms of mathematics or write great works of literature, begging the question: is it reasonable to explain humanity’s greatest scientific discoveries or works of art with the same framework that describes the neural process of training a dog to sit? This talk will explore how fundamental assumptions that underlie the standard RL problem formulation, such as the distinction between the agent and environment, the stationarity of the environment transition dynamics, and the external source of reward, influence the types of behaviour we can expect to observe in our agents. We will conclude with a discussion of alternative assumptions which might be more friendly to the development of agents capable of the open-ended, iterative, and creative intelligent behaviours necessary for pushing the frontier of human knowledge.

• 10:10 - 10:40 (30 minutes) - Coffee Break 

• 10:40 - 11:40 (1 hour) - Panel Discussion

• 11:40 - 12:30 (50 minutes) - Poster Session #1

• 12:30 - 14:00 (1 hour 30 minutes) - Lunch Break

• 14:00 - 14:45 (45 minutes) - Lightning Talks (~ 13 mins for each talk including its Q&A time)

  1. Thinking is Another Form of Control. Josiah P. Hanna and Nicholas E. Corrado. 

     • Awarded: Most Thought-provoking paper.

  2. Analogy making as amortised model construction. David G Nagy, Tingke Shen, Hanqi Zhou, Charley M Wu, and Peter Dayan.

  3. Agent-centric learning: from external reward maximization to internal knowledge curation. Hanqi Zhou, Fryderyk Mantiuk, David G Nagy, and Charley M Wu.

• 14:45 - 15:05 (20 minutes) - Tea Break

• 15:05 - 16:05 (1 hour) - Poster Session #2