Research

My research interest is mainly in natural language processing (NLP), especially language generation.

I am extremely  fascinated by understanding the working mechanism of NLP models.  Although deep neural networks have achieved great success in a variety of NLP tasks, we are still not clear why it works or fails on some cases. The in-transparent working mechanism not only makes the model uncontrollable, but also hinders us to propose more effective neural networks. In my research, I aim to address this issue from three perspectives: model, data, and linguistic information.

I am also interested in the applications of  NLP models in real scenarios. There is still a big gap to make a SOTA model in research work well in the real scenario.  For example, besides high translation quality, human translators may want an engine to avoid repeated translation errors  once they address it at its first occurrence. It is interesting to abstract research problems from real scenarios and make a good influence on the use of techniques in the real world.

About Me

I am a second-year Ph.D. student at NAIST, advised by Prof. Taro Watanabe. During the Master period at NAIST, I had two wonderful internships at Apple Siri and Amazon Translate teams. Before joining NAIST, I worked at Tencent AI Lab as a full-time researcher, mentored by Dr. Lemao Liu and Guoping Huang. I am proud that we released the API of the first Interactive Machine Translation (IMT) in China, namely TranSmart. I got my bachelor degree from Central China Normal University, advised by Prof. Qingxing Dong.

Publications

(*: equal contribution, Topics: 🟦 Text Generation 🟥 Cross-lingual, 🟩 Multi-modal)

Conference

• 🟦 Haoran Yang, Deng Cai, Huayang Li, Wei Bi, Wai Lam, Shuming Shi. A Frustratingly Simple Decoding Method for Neural Text Generation. COLING 2024.

• 🟦 Huayang Li, Tian Lan, Zihao Fu, Deng Cai, Lemao Liu, Nigel Collier, Taro Watanabe, Yixuan Su. Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective. NeurIPS 2023. 

• 🟦 Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li, Jian Li. Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation. NeurIPS 2022.

• 🟦 Huayang Li, Deng Cai, Jin Xu, Taro Watanabe. Residual Learning of Neural Text Generation with n-gram Language Model. Findings of EMNLP 2022.

• 🟦 Wei Bi*, Huayang Li*, Jiacheng Huang. Data Augmentation for Text Generation Without Any Augmented Data. ACL 2021.

• 🟦 Jiannan Xiang, Yahui Liu, Deng Cai, Huayang Li, Defu Lian, and Lemao Liu. Assessing Dialogue Systems with Distribution Distances. Findings of ACL 2021.

• 🟦 Huayang Li, Lemao Liu, Guoping Huang, Shuming Shi. On the Branching Bias of Syntax Extracted from Pre-trained Language Models. Findings of EMNLP 2020.

• 🟥 Jiannan Xiang*, Huayang Li*, Defu Lian, Guoping Huang, Taro Watanabe, Lemao Liu. Visualizing the Relationship Between Encoded Linguistic Information and Task Performance. Findings of ACL 2022. 

• 🟥 Jiannan Xiang, Huayang Li, Yahui Liu, Lemao Liu, Guoping Huang, Defu Lian, Shuming Shi. Investigating Data Variance in Evaluations of Automatic Machine Translation Metrics. Findings of ACL 2022.

• 🟥 Huayang Li, Lemao Liu, Guoping Huang, Shuming Shi. GWLAN: General Word-Level Autocompletion for Computer-Aided Translation. ACL 2021.

• 🟥 Deng Cai, Yan Wang, Huayang Li, Wai Lam, and Lemao Liu. Neural Machine Translation with Monolingual Translation Memory. Outstanding Paper Award of ACL 2021.

• 🟥 Jierui Li, Lemao Liu, Huayang Li, Guanlin Li, Guoping Huang and Shuming Shi. Evaluating Explanation Methods for Neural Machine Translation. ACL 2020.

• 🟥 Tianxiang Zhao, Lemao Liu, Guoping Huang, Huayang Li, Yingling Liu, Guiquan Liu, Shuming Shi. Balancing Quality and Human Involvement: An Effective Approach to Interactive Neural Machine Translation. AAAI 2020.

• 🟩 Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, Deng Cai. Pandagpt: One model to instruction-follow them all. TLLM 2023

• 🟩 Yahui Liu, Marco De Nadai, Deng Cai, Huayang Li, Xavier Alameda-Pineda, Nicu Sebe, Bruno Lepri. Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach. ACMM 2020.

Journal

• 🟥 Huayang Li, Guoping Huang, Deng Cai, Lemao Liu. Neural Machine Translation with Noisy Lexical Constraints. IEEE/ACM TASLP.

Preprint

• 🟦 Huayang Li*, Yixuan Su*, Deng Cai*, Yan Wang*, Lemao Liu*. A Survey on Retrieval-Augmented Text Generation. arXiv 2022.

• 🟦 Shuming Shi, Enbo Zhao, Deng Cai, Leyang Cui, Xinting Huang, Huayang Li. Inferflow: an Efficient and Highly Configurable Inference Engine for Large Language Models. arXiv 2024.

• 🟥 Huayang Li, Deng Cai, Zhi Qu, Qu Cui, Hidetaka Kamigaito, Lemao Liu, Taro Watanabe. Cross-lingual Contextualized Phrase Retrieval. arXiv 2024.

• 🟥 Guoping Huang, Lemao Liu, Xing Wang, Longyue Wang, Huayang Li, Zhaopeng Tu, Chengyan Huang, Shuming Shi. TranSmart: A Practical Interactive Machine Translation System. arXiv 2021.

• 🟩 Huayang Li*, Siheng Li*, Deng Cai*, Longyue Wang, Lemao Liu, Taro Watanabe, Yujiu Yang, Shuming Shi. TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild. arXiv 2023.

Selected Awards & Honors

• Special Researcher, Japan Society for the Promotion of Science (JSPS, 日本学術振興会), 2023

• Outstanding Paper Award of ACL, 2021

Education

• Apr. 2023 - Present: Ph.D., Division of Information Science,  Nara Institute of Science and Technology

• Apr. 2021 - Mar. 2023: M.E., Division of Information Science,  Nara Institute of Science and Technology

• Sep. 2014 - Jun. 2018: B.E., Dept. of Computer Science, Central China Normal University

Research Experience

• Jun. 2022 - Oct. 2022: Applied Scientist Intern, Amazon Translate, with Maria Nadejde, Xing Niu, Stanislas Lauly, and Yogarshi Vyas.

• Oct. 2021 - Jun. 2022: Research Intern, Apple Siri, with Xiaojiang Liu

• Jul. 2018 - Apr. 2021: Researcher, Tencent AI Lab, with Guoping Huang and Lemao Liu.