Welcome~

Yujia Xie

I am a senior researcher at Microsoft Azure Language Pillars, landing language models to the cloud service. Previously, I work in the Microsoft Azure Cognitive Service Research Team, doing research on the large-scale training of multimodal representation models.

I got my Ph.D. degree from Georgia Tech in 2021, advised by Dr. Hongyuan Zha and Dr. Tuo Zhao. Prior to Ph.D. study, I received a bachelor’s degree in 2016 in theoretic and applied mechanics at Special College of Gifted Young (SCGY), University of Science and Technology of China (USTC).

My research interest lies in the fundamental and practical problems in machine learning, with emphasis on

The applicability and adaptivity of large language models
The perception, understanding and reasoning across different modalities
End-to-end learning of large non-differentiable models.

Xie.Yujia000@gmail.com, Google Scholar, Linkedin, CV

Selected Publications

Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles

Shuquan Ye (my intern), Yujia Xie, Dongdong Chen, Yichong Xu, Lu Yuan, Chenguang Zhu, Jing Liao

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023, PDF

Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning

Yujia Xie, Luowei Zhou, Xiyang Dai, Lu Yuan, Nguyen Bach, Ce Liu, Michael Zeng

Advances in Neural Information Processing Systems (NeurIPS) 2022, PDF

REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering

Yuanze Lin (my intern), Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, Lu Yuan

Advances in Neural Information Processing Systems (NeurIPS) 2022, PDF

A Hypergradient Approach to Robust Regression without Correspondence

Yujia Xie*, Yixiu Mao*, Simiao Zuo, Hongteng Xu, Xiaojing Ye, Tuo Zhao, Hongyuan Zha

International Conference on Learning Representations (ICLR) 2021, PDF, code

Differentiable Top-k Operator with Optimal Transport

Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

Conference on Neural Information Processing Systems (NeurIPS) 2020, PDF, code, poster

Meta Learning with Relational Information for Short Sequences

Yujia Xie, Haoming Jiang, Feng Liu, Tuo Zhao, Hongyuan Zha

Conference on Neural Information Processing Systems (NeurIPS) 2019, PDF, code, poster

On Scalable and Efficient Computation of Large Scale Optimal Transport

Yujia Xie, Minshuo Chen, Haoming Jiang, Tuo Zhao, Hongyuan Zha

International Conference on Machine Learning (ICML) 2019, PDF, code, poster

A Fast Proximal Point Method for Computing Wasserstein Distance

Yujia Xie, Xiangfeng Wang, Ruijia Wang, Hongyuan Zha

Conference on Uncertainty in Artificial Intelligence (UAI) 2019, PDF, code

Experience

Sep. 2022 - Present: Senior Researcher at Microsoft, Redmond, Azure Language Pillars
Sep. 2021 - Sep. 2022: Researcher at Microsoft, Redmond, Azure Computer Vision Research Team
Feb. 2021 - May. 2021: Applied Scientist Internship at Amazon, Palo Alto mentored by Dr. Li Zheng and Dr. Danqing Zhang.
Jun. 2020 - Aug. 2020: Quantitative Research Internship at Citadel, Seattle mentored by Dr. Li Deng and Dr. Pusheng Zhang.

Performed fraud detection using textual data from financial documents. Improved AUC significantly over the traditional method.

Sep. 2019 - Apr. 2020: Research Internship at Google, Sunnyvale mentored by Dr. Wei Wei.

Proposed a differentiable and efficient top-k operator, demonstrating significant improvements in image recognition and machine translation.

May 2018 - Aug. 2018: Research Internship at Microsoft, Redmond mentored by Dr. Yi Mao and Dr. Weizhu Chen.

Proposed a conditional self-attention network for query-based summarization tasks. Achieved improvements in ROUGE score by at least 4.

Reviewer of ICML, NeurIPS, AAAI, JMLR
Teaching Assistantship of CS 7641 Machine Learning & CSE 6740 Computational Data Analysis
Research Assistantship (2016 - Present)

Welcome~

Yujia Xie

Selected Publications

Experience

PhD Research