dr. Cheems Wang

Recent News and Publications

• August 15th: I am invited to serve as the Area Chair (Senior PC) for AAMAS 2026. Thanks for the Program Chairs' nomination🎉I am happy to accept it ☺️!

• July 7th: Our paper "Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models?" is now made public! We successfully extend model predictive task sampling to the case with discrete task spaces, and accelerate RL finetuning Large Language Models! 🥳

• May 1st: Our paper "Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments" was accepted to ICML2025, Congratulations to Qu Yun! Want to know how to robustify Meta RL and Domain Randomization without extra agent-environment interactions and accelerate learning process, feel free to find it at https://thu-rllab.github.io/PDTS_project_page/. 

• April 9th: I am invited to serve as the Area Chair for NeurIPS 2025. Thanks for the Program Chairs' nomination🎉yet I fail to work as🥹.

• January 19th: Our paper "Model Predictive Task Sampling for Efficient and Robust Adaptation" is made public! This is the first trial to successfully predict the optimization outcome after any-shot adaptation given some tasks and model predictive task sampling (MPTS) significantly improves the adaptation robustness of multimodal foundation models and sequential decision-making with learning efficiency retained!🥳One of my favorate research outputs I can feel proud of after my graduation from AMLab (Birthplace of VAEs)!

• January 19th: "DynaPrompt: Dynamic Test-Time Prompt Tuning" was accepted to ICLR 2025. Congratulations ot Zehao and I feel honored to collaborate in this Project!

• December 10th: Our paper  "Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning" was accepted to AAAI2025. Congratulations to Yun Qu! Preprint will come soon.

• November 17th: Our paper "Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation" got accepted to KDD2025 Research Track. Congratuations to all collaborators!

• September 26th: Our paper "Doubly Mild Generalization for Offline Reinforcement Learning" got accepted to NeurIPS2024. Preprint will come soon.

• September 26th: Our paper "Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression" got accepted to NeurIPS2024. Preprint will come soon.

• September 26th: Our paper "GO4Align: Group Optimization for Multi-Task Alignment" got accepted to NeurIPS2024. Camera ready version will come soon. [Paper Link, Code Link]

• September 26th: Four papers got accepted to NeurIPS2024. Congratulations to all students I co-superivsed (Cheems as the Second-Author for ALL 🤪)

• July 28th: Our latest work "Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation" was now available! Feel free to read our blog here. This work handles the task distribution shifts between meta-training and meta-testing and proposes to generate explicit task distributions in an adversarial way, increasing the robustness of fast adaptation 😄! Codes will be released and watch our progress! [Paper Link, Blog Link]

• May 1st: The research work "Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation" was accepted to ICML2024, congratulations to all my awesome collaborators! Here, a surrogate back-propagation algorithm is developed together with a variant of Layer Normalization. This work significantly reduces memory usage on Lima and ViT and provides theoretical guarantees. Codes will be released soon!

• April 2024: For researchers interested in Multi-task Learning, you cannot miss this wonderful work🥳, "GO4Align: Group Optimization for Multi-Task Alignment". Here, we align the learning progress across tasks to promote the exploitation of beneficial task-relatedness and achieve SOTA performance in most existing benchmarks. Codes will be released soon!

• October 2023: I joined Tsinghua University and working as a researcher in automation. Thanks for Prof. Xiangyang Ji's supervision.

• September 22nd 2023: Congratulations to Jiayi Shen! Our manuscript "Episodic Multi-Task Learning with Heterogeneous Neural Processes" was accepted to NeurIPS2023 Spotlight (top 3.06% in all submissions), and this work proposes heterogeneous neural processes to exploit task relatedness in multi-task learning. [Paper Link, Code Link]

• July 2023: Many thanks to Max and Herke's nomination and Sara, Yang, Sihang's recommendation, I am honored to be awarded China Computer Federation Multi-agent System Groups (CCF-CMAS) Best Dissertation Prize [中国计算机学会-多智能体系统学组优博论文奖].

• January 21st 2023: Our manuscript "Bridge the Inference Gaps of Neural Processes via Expectation Maximization" was accepted to ICLR2023.

• October 10th 2022: Very pleased that I received NeurIPS Scholar Award, and thanks to the NeurIPS Foundation.

• September 14th 2022: Our paper Learning Expressive Meta-Representations with Mixture of Expert Neural Processes was accepted to NeurIPS2022. Our method applies to few-shot supervised learning and meta reinforcement learning. It is able to handle stochastic processes with mixture components. [Paper Link]

• July 2022: The draft of my Ph.D. Thesis will be finished. 

• July 7th 2022: Delighted to receive ICML2022 Participation Grants.

• May 2022: Our paper Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models and Amortized Policy Search was accepted in ICML2022 Spotlight, and the final version will be released soon. It seems the log-likelihood of defending my Ph.D. thesis this year was increased!

Here a GNN based dynamics model is introduced with superior generalization, and the posterior sampling strategy is used in policy learning without additional policy gradients in new environments. 

Feel free to access our slides link below as a brief introduction to our proposed GSSM and Amortized Meta Model-based Policy Search. (Note that this is the first trial in amortizing task-specific policies in meta model-based policy search as far as we know. The significance is that the use of non-parametric modeling avoids re-training or gradient adaptations of policies in new environments, which might be promising to address data-efficient fast adaptation problems.)

[The latest version can be found here: Paper Link, Slides Link]

• June 1st, 2020 : Our paper entitled "Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables" is accepted in ICML2020. (Supplementary materials refer to the Link.)

 The slides for our ICML presentation are attached to the Link below. We propose a hierarchical neural process to simultaneously identify tasks and capture local correlations in high-dimensional problems.

[Paper Link, Slides Link]