World Models

Workshop Scope

The concept of the "World Model" focuses on how intelligent agents can understand and model the external interactive worlds/environments to improve their decision-making and planning abilities. World models were initially focused on modelling low-level physical quantities and interactions by recurrent neural networks (RNNs). Over time, the "World Models" concept has expanded to real-world simulation (e.g. Sora and Genie) and the generation of complex, realistic, and high-dimensional environments.

This workshop explores classical World Modelling backbones for understanding and modelling the world, such as Transformers, RNNs, state-space models (SSMs), spatial-temporal modelling and causality analysis. Building from these foundational topics, the workshop will also discuss the broader and evolving concept of "World Models" for complex real-world prediction and simulation, like video/text generation and more specific applications like embodied AI, healthcare and sciences. This evolution highlights the growing complexity and capabilities of World Models. By bringing together leading researchers, the workshop will cover both classical and cutting-edge techniques, and discuss how World Models can be applied across a wide range of emerging applications.  Some of the fundamental questions and specific challenges that this workshop aims to address are:

• Understanding the World and Extracting Knowledge.

• World Model Training and Evaluation.

• Scaling World Models Predictions Across Language, Vision, and Control.

• World Models in General Domains: Embodied AI, Healthcare, Natural and Social Sciences, and Beyond.

The workshop covers the widest range of World Models topics, including understanding, modelling, as well as scaling with cutting-edge generative AI. We welcome submissions related to the construction, analysis and applications of world models, such as Model-Based Reinforcement Learning, Causality, Sequential Modelling, Simulation of the Environment, Diffusion Models, Video Generation, Foundation World Models, 2D to 3D, Spatial Intelligence, Robotics, and Embodied AI etc.

We also encourage submissions from the Natural Sciences (e.g., physics, chemistry, biology) and Social Sciences (e.g., pedagogy, virtual sociology simulation) related to world/environment construction in the science domain to offer attendees a more comprehensive perspective. In summary, topics of interest mainly include, but are not limited to:

• Understanding World Rules: Exploring World Models capture environment dynamics; causality understanding; spatial-temporal modelling; model-based RL; and theoretical foundations for environment simulation and prediction. 

• World model training and evaluation: strengths, limitations, and challenges of current modelling architectures (e.g. Transformers, RNNs, and SSMs), training algorithms (autoregressive training, diffusion modelling, RL, and normalizing flow) and dataset construction. 

• Scaling World Models prediction and generation across language, vision, and control: Investigating how integrating visual, auditory, and textual data improves realism World Models.

• World Models in general domains: Exploring World Models in robotics, AI, healthcare, natural and social sciences, and beyond to improve prediction and decision-making. 

• Benchmark, Dataset, and Demonstration about World Models such as environment simulation.

• ...

Invited Speakers & Panelist

Organizing Team

Sponsors