Generative Models for Decision Making

Generative Artificial Intelligence (AI) has made significant advancements in recent years, particularly with the development of large language and diffusion models. These generative models have demonstrated impressive capabilities across various domains, such as text [1-3], image, audio, and video [4, 5, 6,7]. Concurrently, decision making has made significant strides in solving complex sequential decision-making problems with the help of external knowledge sources [8-10]. However, there remains untapped potential in combining generative models with decision making algorithms to tackle real-world challenges, particularly to improve sample efficiency of tabula rasa training by introducing priors from related domains such as visual question-answering, image captioning and image generation.

This workshop aims to bring together researchers and practitioners from the fields of generative AI and decision making to explore the latest advances, methodologies, and applications. By fostering collaborations between these two domains, we intend to unlock new opportunities for addressing complex problems that lie at the intersection of both fields.

The workshop will cover a wide range of topics, including but not limited to:

• Large Language Models and Decision Making: Exploring how large language models, such as GPT-4 and beyond, can be integrated with decision making algorithms to improve performance on complex sequential decision-making tasks. Moreover, we welcome contributions which study how to make large language model suitable for interactive and embodied settings, be it for planning, reward generation, simulation of the physical world or introducing human priors into decision making via language. Tentative research questions: which benchmarks, evaluation criteria and environments should be developed by the community to assess the utility of large language models for decision making?

• Diffusion Models and Decision Making: Investigating the potential of diffusion models and other generative models for enhancing decision making algorithms for planning, reinforcement learning from pixels, and robotic control. Tentative research questions: can diffusion models be used as physics-aware world models, thus improving the sample efficiency of online decision making methods?

• Sample Efficiency in Decision Making: Discussing techniques for improving sample efficiency in decision making through generative models, enabling the application of decision making in data-constrained environments. Specifically, can generative models trade reward-labelled efficiency by using more unlabelled samples? Tentative research questions: can we use large language model or video prediction models to enable faster learning on complex, open-ended decision making tasks?

• Exploration in Decision Making: Exploring how generative models can facilitate exploration strategies in decision making, especially in high-dimensional and sparse reward settings. For instance, since generative models can efficiently represent parts of the data distribution, it is reasonable to assume that they can also provide an informative learning signal. Tentative research questions: how can pre-trained generative models help decision making agents solve long-horizon, sparse reward  or open-ended tasks without a clear definition of success?

• Transfer Learning in Decision Making with Generative Models: Investigating methods to leverage pre-trained generative models for transfer learning in decision making, enabling agents to adapt to new tasks more efficiently through a deeper understanding of the underlying dynamical system of decision making problems. Tentative research questions: do generative models used for high-level planning or low-level control transfer better to unseen domains than classical decision making methods?

• Inverse Reinforcement Learning and Imitation Learning: Analyzing how generative models can assist IRL/IL algorithms in learning from observed behaviour, or used for data augmentation. Tentative research questions: can generative models capture richer information contained in human demonstrations than existing methods?

Generative AI has led to significant advances in natural language, vision, audio, and video.  Such advances can lead to fundamental changes in decision making, and with the aim for bridging generative AI with the  decision making community from control, planning, and reinforcement learning, we invite submissions in this area including the following topics:

• Studying how generative models can directly be used as decision making agents – i.e. a LLM agent.

• Studying how generative models can algorithmically change the decision making problem – i.e. formulating decision making as reward-conditioned generative modelling or planning as inference on a generative model.

• Studying how the priors in large generative models can enable sample efficiency and effective exploration.

• Studying how generative models can aid the inference of the intent of a set of demonstrations (i.e. inverse reinforcement learning).

• Studying how generative models can enable effective transfer learning.

Submission instructions:

Submissions should follow the official ICLR 2024 LaTeX template . We welcome technical or positional papers of the following formats: short (4 pages excluding references and appendix), and long (9 pages excluding references and appendix). The main paper and the appendix should be submitted as a single PDF file on the submission website. Papers that have previously been published in other conferences will not be accepted. However, submissions of unpublished or ongoing work, such as works under review at ML conferences, are welcome.

Submission link:

All submissions should be done via OpenReview: https://openreview.net/group?id=ICLR.cc/2024/Workshop/GenAI4DM .

Reviewer recruiting form:

If you are interested in reviewing for the workshop, fill out the form: https://forms.gle/hFiLgN91vBVgvnFp9 

Double-blind policy:

 All submissions should be properly anonymized by the authors to remove any identifying information such as author names, affiliations, personalized Github links, etc.

 Please feel free to reach out with any questions about the CFP or the workshop by emailing genai.dm.iclr2024@gmail.com .

Paper submission deadline: February 9th 2024 (Anywhere on Earth)

Decision notifications March 13rd 2024 (released)

Camera-ready paper deadline: N/A since the workshop has no proceedings

Workshop: May 11th 2024

TBD