2022 GAC 3

What is the place of planning?

Authors: 

Malcolm A. MacIver, Northwestern University

Nathaniel D. Daw, Princeton University

German Espinosa, Northwestern University

Jessica B. Hamrick, DeepMind

Mark K. Ho, Princeton University, NYU

A. David Redish, University of Minnesota

Bradly C. Stadie, Northwestern University

Jane X. Wang, DeepMind

Abstract:

Are there problems that fundamentally require planning and cannot be solved via model-free learning? If so, what defines those problems? If not, does this mean that intelligence has, in principle, already been solved by current deep RL methods? The past decade has shown marked growth in interest in reinforcement learning (RL) and deep learning (DL). Model-free methods have allowed computers to achieve super-perhuman performance in complex domains like board-games, and even in real-time execution like Atari games and Starcraft, suggesting that with sufficient experience and computational resources, cognitively simple architectures (e.g., feedforward NN) can learn rich behavioral repertoires. Where does this leave more complex cognitive processes, in particular, model-based planning? Despite being a central concern at the founding of artificial intelligence (AI) and a key topic in psychology/neuroscience, planning plays only a secondary role in modern deep learning approaches and is primarily conceptualized as one of several techniques for reducing sample complexity in tasks that would otherwise be solved with purely model-free methods. This GAC will discuss the evolutionary process that constrained the specific place planning has in nature and if it is possible to use this information to determine planning’s applicable domains in AI.

>>> the workshop recording can be found here <<<

Instructions for CCN community members: 

1. Download the proposal PDF using the button above. 

2. Use the Disqus box below to leave comments and reviews, and discuss the submission. You'll need to create a Disqus account or log into Disqus using a google account or similar in order to leave a comment/review.