Foundation Models for Decision Making

Breakfast

Opening Remarks

Oral Presentations

• 9:00 - 9:15: Is Conditional Generative Modeling all you need for Decision-Making?

• 9:15 - 9:30: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training

• 9:30 - 9:45: VIMA: General Robot Manipulation with Multimodal Prompts

• 9:45 - 10:00: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action

Gabriel Barth-Maron: Gato: A Generalist Agent

Jim Fan: Open-Ended Embodied Agents with Internet-Scale Knowledge

Leslie P. Kaelbling: What does an intelligent robot need to know?

Dorsa Sadigh: Learning and Leveraging Foundation Models in Robotics

Oral Presentations

• 13:00 - 13:15: REACT: Synergizing Reasoning and Acting in Language Models

• 13:15 - 13:30: Generative Pretraining for Black-Box Optimization

• 13:30 - 13:45: In-context Reinforcement Learning with Algorithm Distillation

• 13:45 - 14:00: Large Language Models Are Human-Level Prompt Engineers

Thomas Wolf: Unlocking Foundation Models for Embodied Learning – What Tools Will We Need?

Machel Reid: On using pre-trained language models for reinforcement learning

Deepak Pathak: A Bottom-Up Approach to Robot Learning in the Wild

Dale Schuurmans: Large Foundation Models and Reinforcement Learning

Panel Discussions: On the role of industry and academia in developing foundation models for decision making.

Poster Session II