My name is Hao Cheng .
I'm a researcher at Microsoft Research and Affiliate Faculty at the University of Washington.
Prior to this, I completed my PhD at the University of Washington working with Mari Ostendorf, and got my MSc under the supervision of Dale Schuurmans and Csaba Szepesvári at the University of Alberta. I am a pround member of Sounding Board the 2017 Alexa Prize Winner!
Email (for company related): {my_last_name}.Hao@microsoft.com
Email (others): {my_first_name}cheng@outlook.com
Test-time scaling can be the next thing to unlock the self-improvement in LLMs. Checkout out our recent effort in search-augmented learning and on-the-fly parameter update.
Fail to prompt LLMs to reliably use context for generation? Our paper proposed an effective solution: combining text prompting with attention Ops.
Too costly to inference with MoE models? Keep the diversity when pruning. Check out our MoE pruning method with improved efficiency and mininal performance drop.
Curious about the proper scaling of MoE models with scalable training? Checkout out GRIN with techniques and insights into building Phi-MoE.
Too much cost for handling long-context if there are repeated queries for a common prefix (e.g., long document/conversation)? Try parallel context encoding! Check out PiD, a new method for efficient long-context handling.
How to orchestrate different tools and LMs for various task-solving? Checkout out LLaVA-Plus and OrchestraLLM.
Volunteer Chairs for NAACL 2021
CollabLLM: From Passive Responders to Active Collaborators
Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, Jianfeng Gao.
In Proc. International Conference on Machien Learning (ICML), 2025. ***Outstanding Paper Award***
Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass
Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng.
In Proc. International Conference on Learning Representations (ICLR), 2025.
ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning
Xiao Yu, Baolin Peng, Vineeth Vajipey, Hao Cheng, Michel Galley, Jianfeng Gao, Zhou Yu.
In Proc. International Conference on Learning Representations (ICLR), 2025.
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang, Jianfeng Gao, Chunyuan Li.
In Proc. European Conference on Computer Vision (ECCV), 2024.
Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues?
Bo-Ru Lu, Nikita Haduong, Chia-Hsuan Lee, Zeqiu Wu, Hao Cheng, Paul Koester, Jean Utke, Tao Yu, Noah A. Smith, Mari Ostendorf.
In Proc. Conference on Language Modeling (COLM), 2024.
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, Jianfeng Gao.
In Proc. International Conference on Learning Representations (ICLR), 2024.
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao.
In Proc. of the Neural Information Processing Systems (NeurIPS), 2023.
Augmenting Language Models with Long-Term Memory
Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei.
In Proc. of the Neural Information Processing Systems (NeurIPS), 2023.
Chain-of-Skills: A Configurable Model for Open-domain Question Answering
Kaixin Ma*, Hao Cheng*, Yu Zhang, Xiaodong Liu, Eric Nyberg, Jianfeng Gao. [*Equal contribution]
In Proc. Assoc. for Computational Linguistics (ACL), 2023.
Open Domain Question Answering with A Unified Knowledge Interface [Code]
Kaixin Ma*, Hao Cheng*, Xiaodong Liu, Eric Nyberg, Jianfeng Gao. [*Equal contribution]
In Proc. Assoc. for Computational Linguistics (ACL), 2022.
UnitedQA: A Hybrid Approach for Open Domain Question Answering [Code]
Hao Cheng*, Yelong Shen*, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao. [*Equal contribution]
In Proc. Assoc. for Computational Linguistics (ACL), 2021.
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Yu Gu*, Robert Tinn*, Hao Cheng*, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon. 2021 [*Equal contribution]
ACM Transactions on Computing for Healthcare --- ***Best Paper Award***
[Instructor][Grad] E596/LING: 580 Conversational AI (course webpage) [Spring 2019]
[TA][Grad] E596/LING 580: Conversational AI (course webpage) [Spring 2018]
[TA] [Grad] EE511: Introduction to Statistical Learning (course webpage) [Winter 2018]
[TA] [Undergrad] EE 235: Continuous-time Linear Systems [Autumn 2017]
[TA] [Undergrad] EE 341: Discrete-Time Linear Systems [Spring 2016]