Co-Evolving Latent Action World Models

 A. Performance of CoLA-World on A Different Video Generation Backbone (Wan2.1)

To further verify the architectural universality and scalability of the CoLA-World joint training framework, we extend our experiments by replacing the underlying video generation backbone from OpenSora to the Wan2.1 model. 

Crucially, to ensure a controlled comparison, we maintain an identical experimental protocol to Section 4.2 in the main paper. We finetune Wan2.1-T2V-1.3B into action-conditioned video generation model (world model). The mechanism for injecting latent action conditions into the Wan2.1 diffusion transformer (DiT) architecture mirrors the strategy employed in our OpenSora implementation, with action tokens integrated via AdaLN. For joint training, we warmup the IDM and the quantizer for 5K steps until the metrics of the codebook stabilize at high levels (e.g., high utilization). We reuse the LAM trained for 30K steps in Section 4.2 in the main paper for 2-stage methods. Quantitative results are detailed in the table below. Our joint training pipeline outperforms or achieves comparable results to 2-stage baselines even when assigned with a much lower training budget (45K v.s 70K). This ablation demonstrates that our joint training paradigm is model-agnostic and can readily benefit from advancements in foundation video generation models.

 C. Long Horizon Autoregressive Video Prediction

We evaluated long-horizon autoregressive real-action video prediction across all four Libero suite datasets. We use the same WM checkpoints, action adapters and protocals as the Libero experiment in Sec 4.4, where we finetune the WM using the whole Libero dataset and evaluate on each of the four suites. To mitigate compounding errors, each generating step conditions on the 5 most recent states plus the initial anchor frame and generate the next 2 states. Therefore, generating a full trajectory (averaging about 150 frames) requires about 25 autoregressive steps, as we use a temporal downsample rate of 3 for Libero's LAM. We use only 5 denoising steps and disable classifier-free guidance to accelerate inference. Results below show that CoLA-World consistently outperforms the 2-stage baseline across most suites and metrics. This confirms our co-trained, collapse-resistant LAM and WM successfully transfer their short-sequence prediction superiority into stable long-horizon rollouts. Admittedly, the absolute long-horizon prediction performance remains modest, especially on Libero-Long. This is because our current pretraining pipeline does not yet incorporate specific optimizations tailored for autoregressive generation (e.g., context noise injection or extended training context lengths). Integrating these standard techniques presents a straightforward avenue to further enhance long-horizon stability in future work.