Yu Wu
Senior Researcher @ Microsoft Research Asia
Bio
I am a senior researcher at the Natural Language Computing Group, Microsoft Research Asia (MSRA). I obtained my B.S. degree and Ph.D. at Beihang University under the mentorship of Prof. Ming Zhou and Prof Zhoujun Li. I have a long history with MSRA, having been an intern there from 2013 to 2019, and officially joined MSRA in June 2019. I am an expert in building open-domain chatbots. Currently, I focus on researching end-to-end speech recognition and speech pre-training.
My research career has been marked by numerous accomplishments, including publishing more than 50 papers in top-tier conferences and journals such as ACL, EMNLP, NIPS, AAAI, IJCAI, and ICML. One of them was nominated as InterSpeech best student paper. Additionally, I am the first author of state-of-the-art models such as VALL-E, a zero-shot text-to-speech model that can duplicate anyone’s voice in three seconds, and WavLM, the state-of-the-art full stack speech understanding model. My technologies have also been integrated into various Microsoft products, including Windows, Azure, and Teams.
I have also won numerous awards such as Microsoft Fellowship (10 winners in Asia), Adept Mind Scholarship (30 winners in the world), and am also recognized as top 50 Chinese Young AI Researchers by Baidu Scholar in 2022.
Email: wumark at 126 dot com
News
BEATs is accepted by ICML 2023 (Oral)
VALL-E wins the UNESCO Netexplo Innovation Award 2023
Projects
Speech pre-training: July 2021 - present
VALL-E:
A language model-based TTS system, which could clone your voice with a 3-second recording, wins the UNESCO Netexplo Innovation Award 2023 (top 10 out of over 3000 innovations).
WavLM:
Ranks 1st in the SUPERB leaderboard and SLT2022 SUPERB Challenge.
Ranks 1st on VoxSRC 2021 speaker verification permanent leaderboard.
Integrate into the official Torchaudio
BEATs
BEATs ranks 1st in the AudioSet, Balanced AudioSet , and ESC-50 audio classification leaderboard.
On-device speech recognition: July 2019 - Mar 2023
Real-Time Transformer based Speech Recognition
Conversation as a Platform, Microsoft (XiaoIce, Rinna): March 2015 - Dec 2019
Core retrieval and generation algorithm
Honors and Awards
Top 10 most significant Innovations, Netexplo, 2023
Global Top 50 Chinese Young Scholars in NLP, Baidu, 2022
InterSpeech Best Student Paper Nomination 2021
AdeptMind ScholarShip 2018
Microsoft Research Asia Ph.D. Fellowship, 2017
National Scholarship, Beihang University, 2015, 2018
Working Experiences
March 2021 - Now, Senior Researcher, Natural Language Computing Group, Microsoft Research Asia.
June. 2019 - March 2021, Researcher, Natural Language Computing Group, Microsoft Research Asia.
Internship Experiences
July. 2013 - Dec. 2017, Natural Language Computing Group, Microsoft Research Asia. Mentor: Wei Wu
Jan. 2018 - Aug. 2018, Natural Language Computing Group, Microsoft Research Asia. Mentor: Furu Wei
Sept. 2018 - May. 2019, Natural Language Computing Group, Microsoft Research Asia. Mentor: Shujie Liu
Students
Guangyu Chen (Ph.D. candidate, Renmin University)
Yuang Li (Master, University of Cambridge -> Huawei)
Xun Gong (Ph.D. candidate, Shanghai Jiaotong University)
Leyang Cui (Ph.D. Zhejiang University -> Senior Researcher@Tencent AI Lab)
Sanyuan Chen (Ph.D. candidate, Harbin Institute of Technology)
Chengyi Wang (Ph. D. candidate, Nankai University -> ByteDance)
Publications
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, Wanxiang Che, Xiangzhan Yu, Furu Wei. BEATs: Audio Pre-Training with Acoustic Tokenizers ICML 2023 (Oral)
Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei. Magneto: A Foundation Transformer ICML 2023
Zhuo Chen, Naoyuki Kanda, Jian Wu, Yu Wu, Xiaofei Wang, Takuya Yoshioka, Jinyu Li, Sunit Sivasankaran, Sefik Emre Eskimez Speech separation with large-scale self-supervised learning ICASSP 2023
Quchen Fu, Szu-Wei Fu, Yaran Fan, Yu Wu, Zhuo Chen, Jayant Gupchup, Ross Cutler Real-time Speech Interruption Analysis: From Cloud to Client Deployment ICASSP 2023
Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian LongFNT: Long-form Speech Recognition with Factorized Neural Transducer ICASSP 2023
Hyungchan Song, Sanyuan Chen, Zhuo Chen, Yu Wu, Takuya Yoshioka, Min Tang, Jong Won Shin, Shujie Liu Exploring WavLM on Speech Enhancement SLT 2023
Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie LIU, Brian Mak Two-Stream Network for Sign Language Recognition and Translation NeurlPS2022
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Zhuo Chen, Peidong Wang, Gang Liu, Jinyu Li, Jian Wu, Xiangzhan Yu, Furu Wei Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition? InterSpeech 2022
Shuo Ren, Shujie Liu, Yu Wu, Long Zhou, Furu Wei Speech Pre-training with Acoustic Piece InterSpeech 2022
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya YoshiokaStreaming Speaker-Attributed ASR with Token-Level Speaker Embeddings InterSpeech 2022
Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka Streaming Multi-Talker ASR with Token-Level Serialized Output Training InterSpeech 2022
Chengyi Wang, Yu Wu, Sanyuan Chen, Shujie Liu, Jinyu Li, Yao Qian, Zhenglu Yang, Self-Supervised Learning for speech recognition with Intermediate Layer Supervision ICASSP 2022
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu, UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training ICASSP 2022
Yiming Wang, Jinyu Li, Heming Wang, Yao Qian, Chengyi Wang, Yu Wu Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition ICASSP 2022
Zhengyang Chen, Sanyuan Chen, Yu Wu, Yao Qian, Chengyi Wang, Shujie Liu, Yanmin Qian, Michael Zeng, Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification ICASSP 2022
Leyang Cui, Yu Wu, Shujie Liu and Yue Zhang A Knowledge Enhanced Fine-Tuning Method for Better Handling Unseen Entity in Dialogue Generation. EMNLP 2021
Jia-Chen Gu, Zhenhua Ling, Yu Wu, Quan Liu, Zhigang Chen and Xiaodan Zhu Detecting Speaker Personas from Conversational Texts EMNLP 2021
Zhong Meng, Yu Wu, Naoyuki Kanda, Liang Lu, Xie Chen, Guoli Ye, Eric Sun, Jinyu Li and Yifan Gong Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition InterSpeech 2021
Eric Sun, Jinyu Li, Zhong meng, Yu Wu, Jian Xue, Shujie Liu and Yifan Gong Improving Multilingual Transformer Transducer Models by Reducing Language Confusions InterSpeech 2021
Naoyuki Kanda, Guoli Ye, Yu Wu, Yashesh Gaur, Xiaofei Wang, Zhong Meng, Zhuo Chen and Takuya Yoshioka Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone InterSpeech 2021
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Takuya Yoshioka, Shujie Liu, Jinyu Li and Xiangzhan Yu Ultra Fast Speech Separation Model with Teacher Student Learning InterSpeech 2021
Jian Wu, Zhuo Chen, Sanyuan Chen, Yu Wu, Takuya Yoshioka, Naoyuki Kanda, Shujie Liu and Jinyu Li Investigation of Practical Aspects of Single Channel Speech Separation for ASR InterSpeech 2021
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data ICML 2021
Leyang Cui, Yu Wu, Jian Liu, Sen Yang and Yue Zhang Template-Based Named Entity Recognition Using BART ACL 2021 findings
Leyang Cui, Sijie Cheng, Yu Wu, Yue Zhang. Does BERT Solve Commonsense Task via Commonsense Knowledge? ACL 2021 findings
Xie Chen*, Yu Wu*, Zhenghao Wang, Shujie Liu, Jinyu Li. Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset ICASSP 2021 (* Equal Contribution)
Sanyuan Chen, Yu Wu, Zhuo Chen, Jian Wu, Jinyu Li, Takuya Yoshioka, Chengyi Wang, Shujie Liu, Ming Zhou Continuous Speech Separation with Conformer ICASSP 2021
Sanyuan Chen, Yu Wu, Zhuo Chen, Takuya Yoshioka, Shujie Liu, Jinyu Li, Xiangzhan Yu. Don't shoot butterfly with rifles: Multi-channel Continuous Speech Separation with Early Exit Transformer ICASSP 2021
Xiong Xiao, Naoyuki Kanda, Zhuo Chen, Tianyan Zhou, Takuya Yoshioka, Sanyuan Chen, Yong Zhao, Gang Liu, Yu Wu, Jian Wu, Shujie Liu, Jinyu Li, Yifan Gong, Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020 ICASSP 2021
Yunli Wang, Lili Mou, Yu Wu, Zhoujun Li. Formality Style Transfer with Shared Latent Space. COLING 2020
Chengyi Wang, Yu Wu, Shujie Liu, Jinyu Li, Liang Lu, Guoli Ye, Ming Zhou. Low Latency End-to-End Streaming Speech Recognition with a Scout Network. InterSpeech 2020 [Best student paper nomination]
Jinyu Li, Yu Wu, Yashesh Gaur, Chengyi Wang, Rui Zhao, Shujie Liu . On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition. InterSpeech 2020
Chengyi Wang, Yu Wu, Yujiao Du, Jinyu Li, Shujie Liu, Liang Lu, Shuo Ren, Guoli Ye, Sheng Zhao, Ming Zhou Semantic Mask for Transformer based End-to-End Speech Recognition. InterSpeech 2020
Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang and Ming Zhou. MuTual: A Dataset for Multi-Turn Dialogue Reasoning. ACL 2020
Shuo Ren, Yu Wu, Shujie Liu, Ming Zhou and Shuai Ma. A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation. ACL 2020 (short paper)
Chengyi Wang, Yu Wu, Shujie Liu, Zhenglu Yang, Ming Zhou. Curriculum Pre-training for End-to-End Speech Translation. ACL 2020
Chengyi Wang, Yu Wu, Shujie Liu, Zhenglu Yang, Ming Zhou. Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation. AAAI 2020
Yu Wu, Yunli Wang, Shujie Liu. A Dataset for Low-Resource Stylized Sequence-to-Sequence Generation. AAAI 2020
Naihan Li, Yanqing Liu, Yu Wu, Shujie Liu, Sheng Zhao, Ming Liu and Ming Zhou. A Robust Transformer based Text-to-Speech Model . AAAI 2020
Kun Zhou, Kai Zhang, Yu Wu, Shujie Liu and Jingsong Yu. Unsupervised Context Rewriting for Open Domain Conversation. EMNLP 2019
Shuo Ren, Yu Wu, Shujie Liu, Ming Zhou and Shuai Ma. Explicit Cross-lingual Pre-training for Unsupervised Machine Translation. EMNLP 2019
Yunli Wang, Yu Wu, Lili Mou, Zhoujun Li, Wenhan Chao. An Empirical Study of Harnessing Pre-Trained Neural Networks with Rules for Formality Style Transfer. EMNLP 2019 (short paper)
Bao, Hangbo, Shaohan Huang, Furu Wei, Lei Cui, Yu Wu, Chuanqi Tan, Songhao Piao, and Ming Zhou. Neural Melody Composition from Lyrics. NLPCC 2019
Shaohan Huang, Yu Wu, Furu Wei, Ming Zhou Dictionary-Guided Editing Networks for Paraphrase Generation. AAAI 2019
Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Ming Zhou, Zhoujun Li, Response Generation by Context-aware Prototype Editing. AAAI 2019 code
Jun Chen, Yu Wu, Xiaoming Zhang, Zhao Yan and Zhoujun Li. Keyphrase Generation with Correlation Constraints. EMNLP 2018
Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou. Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots. ACL 2018 (short paper)
Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou. Dynamic vocabulary for Response Generation. AAAI 2018
Yu Wu, Wei Wu, Can Xu, Zhoujun Li, Ming Zhou. Knowledge Enhanced Hybrid Neural Network for Text Matching. AAAI 2018
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma. Hierarchical Recurrent Attention Network for Response Generation. AAAI 2018
Yu Wu, Wei Wu, Chen Xing, Ming Zhou, Zhoujun Li, Sequential Match Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots. ACL 2017 code
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma Topic Aware Neural Response Generation. AAAI 2017
Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou, Improving Recommendation of Tail Tags for Questions in Community Question Answering. AAAI 2016
Chaozhuo Li, Yu Wu, Wei Wu, Chen Xing, Zhoujun Li, Ming Zhou, Detecting Context Dependent Messages in a Conversational Environment. COLING 2016
Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou, Mining Query Subtopics from Questions in Community Question Answering. AAAI 2015
Journal Papers
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu *, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Micheal Zeng, Furu Wei WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing IEEE Journal of Selected Topics in Signal Processing * Corresponding author
Yu Wu, Wei Wu, Chen Xing, Can Xu, Ming Zhou, Zhoujun Li, A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots. Computational Linguistics
Yu Wu, Wei Wu,Zhoujun Li, Ming Zhou, Response Selection with Topic Clues for Retrieval-based Chatbots. NeuroComputing
Workshop Papers
WenZheng Feng, Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou,Beihang-MSRA at SemEval-2017 Task 3: A Ranking System with Neural Matching Features for Community Question Answering. SemEval@ACL2017
Dejian Yang, Yu Wu, Zhoujun Li, Beihang at the NTCIR-13 STC-2 Task. NTCIR-13
Professional services
Reviewer: AAAI, ACL, CL, EMNLP, ACL, AACL, TKDE, Neural Computation, TASLP
Editor: 《人工智能导论》 中国科学技术出版社 a textbook for Chinese undergraduate students