Welcome


My name is Hao Cheng (程 浩 in Chinese).

I'm a researcher at Microsoft Research. Prior to this, I completed my PhD at the University of Washington working with Mari Ostendorf, and got my MSc under the supervision of Dale Schuurmans and Csaba Szepesvári at the University of Alberta.


Research Interest:

In general, my research interest centers around natural language processing and machine learning


Our team Sounding Board is the 2017 Alexa Prize Winner!

(For details and media coverage, check out more on this link)

Updates:

  • Check out our work on proper benchmarking for few-shot NLU (beyond classification) with pretrained language models (BERT, RoBERTa, DeBERTa, T5, GPT3).

  • Our system, UnitedQA, ranked the 1st place based on the automatic evaluation at the NeurIPS 2020 EfficientQA Competition. Please see our paper for more details.

  • We released our PubmedBERT abstract and full-text, state-of-the-art pretrained langauge models for a wide range of biomedical tasks. Please see our paper for more details.


Papers [Google Scholar]

[Preprint]

Open Domain Question Answering over Virtual Documents: A Unified Approach for Data and Text

Kaixin Ma*, Hao Cheng*, Xiaodong Liu, Eric Nyberg, Jianfeng Gao. [*Equal contribution]

Knowledge-Rich Self-Supervised Entity Linking

Sheng Zhang*, Hao Cheng*, Shikhar Vashishth*, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon. [*Equal contribution]

Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing

Robert Tinn*, Hao Cheng*, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon. [*Equal contribution]

Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang.


[2021]

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Subhabrata Mukherjee, Xiaodong Liu, Guoqing Zheng, Saghar Hosseini, Hao Cheng, Ge Yang, Christopher Meek, Ahmed Awadallah, Jianfeng Gao.

In Proc. of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks), 2021.

Dialogue State Tracking with a Language Model using Schema-Driven Prompting

Chia-Hsuan Lee, Hao Cheng, Mari Ostendorf.

In Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2021.

Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature

Yu Wang*, Jinchao Li*, Tristan Naumann*, Chenyan Xiong, Hao Cheng, Robert Tinn, Cliff Wong, Naoto Usuyama, Richard Rogahn, Zhihong Shen, Yang Qin, Eric Horvitz, Paul N. Bennett, Jianfeng Gao, and Hoifung Poon. [*Equal contribution]

In Proc. of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD '21)

UnitedQA: A Hybrid Approach for Open Domain Question Answering

Hao Cheng*, Yelong Shen*, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao. [*Equal contribution]

In Proc. Assoc. for Computational Linguistics (ACL), 2021.

Posterior Differential Regularization with f-divergence for Improving Model Robustness

Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao.

In Proc. Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2021.

Targeted Adversarial Training for Natural Language Understanding

Lis Pereira*, Xiaodong Liu*, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi.

In Proc. Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2021. [*Equal contribution]

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

Yu Gu*, Robert Tinn*, Hao Cheng*, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon. 2021 [*Equal contribution]

ACM Transactions on Computing for Healthcare


[2020]

Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering

Hao Cheng, Ming-Wei Chang, Kenton Lee, Kristina Toutanova.

In Proc. Assoc. for Computational Linguistics (ACL), 2020

The microsoft toolkit of multi-task deep neural networks for natural language understanding

Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao.

In Proc. Assoc. for Computational Linguistics (ACL), 2020

Adversarial training for large neural language models

Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao. 2020


[Before 2020]:

A Dynamic Speaker Model for Conversational Interactions

Hao Cheng, Hao Fang, Mari Ostendorf.

In Proc. Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019.

Improving Span-based Question Answering Systems with Coarsely Labeled Data

Hao Cheng, Ming-Wei Chang, Kenton Lee, Ankur Parikh, Michael Collins, Kristina Toutanova. arXiv preprint arXiv:1811.02076, 2018

Sounding Board: A User-Centric and Content-Driven Social Chatbot

Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A Smith, Mari Ostendorf.

In Proc. Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), demo, 2018.

Sounding Board–University of Washington’s Alexa Prize Submission

Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarten Sap, Mari Ostendorf, Yejin Choi, Noah A Smith.

In Alexa Prize Proceedings, 2017.

A Factored Neural Network Model for Characterizing Online Discussions in Vector Space

Hao Cheng, Hao Fang, Mari Ostendorf.

In Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2017.

Bi-directional Attention with Agreement for Dependency Parsing

Hao Cheng, Hao Fang, Xiaodong He, Jianfeng Gao, Li Deng.

In Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2016.

Learning Latent Local Conversation Modes for Predicting Community Endorsement in Online Discussions.

Hao Fang, Hao Cheng, Mari Ostendorf.

In Proc. the 4th International Workshop on Natural Language Processing for Social Media (SocialNLP), 2016.

Scalable and Sound Low-Rank Tensor Learning

Hao Cheng, Yaoliang Yu, Xinhua Zhang, Eric Xing, Dale Schuurmans.

In Proc. Conf. Artificial Intelligence and Statistics (AISTATS), 2016.

Open-Domain Name Error Detection using a Multi-Task RNN.

Hao Cheng, Hao Fang, Mari Ostendorf.

In Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2015.

Language Models for Image Captioning: The Quirks and What Works.

Jacob Devlin, Hao Cheng, Hao Fang, Saurabh Gupta, Li Deng, Xiaodong He, Geoffrey Zweig, Margaret Mitchell.

In Proc. Assoc. for Computational Linguistics (ACL), 2015.

Approximate Low-Rank Tensor Learning

Yaoliang Yu, Hao Cheng, Xinhua Zhang.

In Proc. NIPS Workshop on Optimization for Machine Learning, 2014.

Convex Relaxations of Bregman Divergence Clustering

Hao Cheng, Xinhua Zhang, Dale Schuurmans.

In Proc. Conf. Uncertainty Artificial Intelligence (UAI), 2013.

Convex Two-layer Modeling

Ozlem Aslan, Hao Cheng, Xinhua Zhang, Dale Schuurmans.

In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2013.

Characterizing the Representer Theorem

Yaoliang Yu, Hao Cheng, Dale Schuuramns, Csaba Szepesvári.

In Proc. International Conference on Machine Learning (ICML), 2013.


Code

Github


Teaching @ UW

[Instructor][Grad] E596/LING: 580 Conversational AI (course webpage) [Spring 2019]

[TA][Grad] E596/LING 580: Conversational AI (course webpage) [Spring 2018]

[TA] [Grad] EE511: Introduction to Statistical Learning (course webpage) [Winter 2018]

[TA] [Undergrad] EE 235: Continuous-time Linear Systems [Autumn 2017]

[TA] [Undergrad] EE 341: Discrete-Time Linear Systems [Spring 2016]


Professional Service

Organizing Committee

Volunteer Chairs for NAACL 2021

Reviewer

I'm currently reviewing for ARR.

Previously reviewed for ACL (2017-2021), EMNLP (2019-2021), NAACL (2019, 2021), AACL (2020), COLING (2018), IJCAI (2015).