AI That Keeps Up

NeurIPS 2025 Workshop on Continual and Compatible Foundation Model Updates (CCFM)

Sunday, December 7th 2025, 8am-5pm
San Diego Convention Center, Upper Level Room 25ABC
Accepted Papers on OpenReview - NeurIPS Event Page

Foundation models, despite their impressive capabilities, face a critical challenge: they naturally become outdated. Trained on vast datasets, frequently updating these models is expensive. Crucially, these challenges extend beyond the scope of studies in traditional continual learning, as foundation models require rapid and scalable adaptation to dynamic global changes and the emergence of both generalized and specialized tasks. This workshop addresses the urgent need for up-to-date foundation models. We invite researchers to explore cost-effective methods for frequent updates and adaptation, minimizing forgetting and deterioration, ensuring a consistent user experience, and designing dynamic evaluations that remain relevant as models evolve.

Summary

Large Foundation Models (FMs) have achieved remarkable capabilities in multiple modalities, including vision and language [10, 1, 11, 4, 12, 8]. FMs are trained on large datasets collected over many years from the web and other sources at significant costs [9, 5]. Pretrained and fine-tuned FMs naturally become outdated over time, presenting several challenges in their development cycle. The proposed workshop addresses the current and future need for up-to-date FMs. The importance of this topic becomes more evident with the release of second, third, or fourth generation of FMs such as GPT-4.1, Gemini-2.5, and Llama-4.

Our workshop invites contributions in multiple categories including efficient continual learning at scale. Traditional continual learning has focused primarily on task-continual or domain-continual learning, where a model is expected to adapt to new tasks that are well-defined in advance [3, 6]. In reality, we require time-continual learning where the world changes constantly and gradually, without clear boundaries, due to factors like global events, technological advancements, and shifts in societal norms [13,14].

We welcome contributions toward frequent backward-compatible updates and the exploration of theoretical or empirical trade-offs between compatibility and performance. From a user’s perspective, an FM update must maintain backward compatibility [15,16]. This means the model should not regress by giving incorrect responses to prompts that previous versions answered correctly—an important but less studied topic. Backward compatibility is expected whether an FM is used directly as an AI assistant [2] or accessed via an API in a larger ML system, e.g., as an evaluator or data synthesizer [7].

Dynamic evaluation of FMs is another important category of topics in the proposed CCFM workshop. As the world changes, prior evaluations need to be updated [13,14]. Important directions include design of novel evaluations that automatically evolve and adapt as well as tackling train/test data contamination, which can occur when new data used for evaluation is nearly identical to the data used for training, leading to overestimation of model performance.

Speakers

Zeynep Akata
Technical University of Munich

Rahaf Aljundi
Toyota Motor Europe

Christopher Kanan
University of Rochester

Irina Rish
Université de Montréal & Mila

Ludwig Schmidt
Stanford & Anthropic

Yiran Huang
Technical University of Munich

Vaggelis Dorovatas
Toyota Motor Europe

News

2025/11/11: Accepted papers are publicly available on OpenReview.
2025/09/01: Submission Deadline is extended to September 2 AOE.
2025/08/22: Submission Deadline is extended to September 1 AOE.
2025/07/22: Submissions are now open at OpenReview.
2025/07/04: Good news! The CCFM Workshop is officially accepted for NeurIPS 2025 and will be held in San Diego!
2025/05/29: Workshop proposal submitted!

Information for Authors

More info in our Call for Papers

Contact

For any questions or comments about this workshop, please reach out to:

ccfm-neurips2025@googlegroups.com

References

[1] Alayrac, Jean-Baptiste, et al. "Flamingo: a visual language model for few-shot learning." Advances in neural information processing systems 35 (2022): 23716-23736.
[2] Bansal, Gagan, et al. "Beyond accuracy: The role of mental models in human-AI team performance." Proceedings of the AAAI conference on human computation and crowdsourcing. Vol. 7. 2019.
[3] Cossu, Andrea, et al. "Is class-incremental enough for continual learning?." Frontiers in Artificial Intelligence 5 (2022): 829842.
[4] Team, Gemma, et al. "Gemma: Open models based on gemini research and technology." arXiv preprint arXiv:2403.08295 (2024).
[5] Li, Jeffrey, et al. "DataComp-LM: In search of the next generation of training sets for language models." Advances in Neural Information Processing Systems 37 (2024): 14200-14282.
[6] Lin, Zhiqiu, et al. "The CLEAR Benchmark: Continual LEArning on Real-World Imagery." Thirty-fifth conference on neural information processing systems datasets and benchmarks track (round 2). 2021.
[7] Ma, Wanqin, Chenyang Yang, and Christian Kästner. "(why) is my prompt getting worse? Rethinking regression testing for evolving llm apis." Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering-Software Engineering for AI. 2024.
[8] Achiam, Josh, et al. "GPT-4 Technical Report." arXiv preprint arXiv:2303.08774 (2023).
[9] Penedo, Guilherme, et al. "The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only." arXiv preprint arXiv:2306.01116 (2023).
[10] Radford, Alec, et al. "Learning transferable visual models from natural language supervision." International conference on machine learning. PmLR, 2021.
[11] Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
[12] Touvron, Hugo, et al. "Llama 2: Open Foundation and Fine-Tuned Chat Models" arXiv preprint arXiv:2307.09288 (2023).
[13] Garg, Saurabh, et al. "TiC-CLIP: Continual Training of CLIP Models." International Conference on Learning Representations (ICLR) (2024).
[14] Li, Jeffrey, et al. "TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
[15] Jaeckle, Florian, et al. "FastFill: Efficient Compatible Model Update." International Conference on Learning Representations (ICLR) (2023).
[16] Echterhoff, Jessica Maria, et al. "MUSCLE: A Model Update Strategy for Compatible LLM Evolution." Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.