Publications

  • Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang, Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention [PDF]

  • Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang and Zicheng Liu, Playing Lottery Tickets with Vision and Language [PDF][AAAI' 22]

  • Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li, Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models [PDF][Leaderboard] [NeurIPS' 21]

  • Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang The Elastic Lottery Ticket Hypothesis [PDF] [NeurIPS' 21]

  • Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim, NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset [PDF] [Findings of EMNLP' 21]

  • Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng Want To Reduce Labeling Cost? GPT-3 Can Help [PDF] [Findings of EMNLP' 21]

  • Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training [PDF] [CVPR' 21]

  • Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee On Orthogonality Constraints for Transformers [PDF] [ACL' 21]

  • Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang and Jingjing Liu EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets [PDF] [Code] [ACL' 21]

  • Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, Jingjing Liu LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval [PDF] [NAACL' 21]

  • Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Yuwei Fang, Siqi Sun, Yu Cheng and Jingjing Liu Cluster-Former: Clustering-based Sparse Transformer for Question Answering [PDF] [Findings of ACL' 21] (Leaderboard #1 on NaturalQuestions as of Sep. 27, 2020)

  • Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [PDF] [Code] [Slides] [ICLR' 21] (Leaderboard #1 on XTREME and XGLUE as of Sep. 8, 2020)

  • Yuwei Fang*, Shuohang Wang*, Zhe Gan, Siqi Sun and Jingjing Liu FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding [PDF] [Code] [Blog] [AAAI' 21] (Leaderboard #1 on XTREME and XGLUE as of Sep. 8, 2020)

  • Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang and Jingjing Liu Contrastive Distillation on Intermediate Representations for Language Model Compression [PDF] [Code] [Blog] [EMNLP' 20]

  • Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang and Jingjing Liu Cross-Thought for Sentence Encoder Pre-training [PDF] [Code] [EMNLP' 20]

  • Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, Bo Li T3: Tree-Autoencoder Regularized Adversarial Text Generation for Targeted Attack [PDF] [EMNLP' 20]

  • Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, and Jingjing Liu Multi-Fact Correction in Abstractive Text Summarization [PDF] [Blog] [EMNLP' 20]

  • Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, and Jingjing Liu Hierarchical Graph Network for Multi-hop Question Answering [PDF] [EMNLP' 20] (Leaderboard #1 on HotpotQA as of Dec. 1st, 2019)

  • Shuohang Wang, Yunshi Lan, Yi Tay, Jing Jiang, Jingjing Liu Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching [PDF] [AAAI' 20]

  • Yunshi Lan, Shuohang Wang, Jing Jiang Knowledge Base Question Answering With a Matching-Aggregation Model and Question-Specific Contextual Relations. [PDF] [TASLP' 19]

  • Yi Tay, Shuohang Wang, Luu Anh Tuan, Jie Fu, Minh C Phan, Xingdi Yuan, Jinfeng Rao, Siu Cheung Hui, Aston Zhang Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives. [PDF] [ACL' 19]

  • Yi Tay, Aston Zhang, Luu Anh Tuan, Jinfeng Rao, Shuai Zhang, Shuohang Wang, Jie Fu, Siu Cheung Hui Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks. [PDF] [ACL' 19]

  • Shuohang Wang, Sheng Zhang, Yelong Shen, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Jing Jiang Unsupervised Deep Structured Semantic Models for Commonsense Reasoning. [PDF] [NAACL' 19]

  • Shuohang Wang, Mo Yu, Shiyu Chang, Jing Jiang A Co-matching Model for Multi-choice Reading Comprehension. [PDF] [code] [ACL' 18, short]

  • Shuohang Wang, Mo Yu, Jing Jiang, Wei Zhang, Xiaoxiao Guo, Shiyu Chang, Zhiguo Wang, Tim Klinger, Gerald Tesauro and Murray Campbell Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering. [PDF] [code][ICLR' 18]

  • Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerald Tesauro, Bowen Zhou, and Jing Jiang R3: Reinforced Ranker-Reader for Open-Domain Question Answering. [PDF] [code] [AAAI' 18]

  • Shuohang Wang and Jing Jiang A Compare-Aggregate Model For Matching Text Sequence. [PDF] [code] [ICLR' 17]

  • Shuohang Wang and Jing Jiang. Machine Comprehension Using Match-LSTM and Answer Pointer. [PDF] [code] [ICLR' 17]

  • Shuohang Wang and Jing Jiang. Learning Natural Language Inference with LSTM. [PDF] [slides] [bib] [code] [NAACL'16]