Yihe Deng
About me
I'm currently a 3rd-year Ph.D. student at Department of Computer Science, University of California, Los Angeles (UCLA), where I am fortunate to be advised by Prof. Wei Wang. Previously I received my B.Sc. from Department of Math, UCLA, and M.Sc. from Department of Computer Science, UCLA. During that time, I've been an undergraduate researcher at UCLA-NLP group with Prof. Kai-Wei Chang.
My research interests focus on Large Language Models (LLMs). Specifically, I'm interested in aspects including prompting, fine-tuning techniques, efficient inference and hallucinations of LLMs. I also work on robustness and multi-modal learning.
News
(2024/02) Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance has been released on arXiv 🚀
(2024/01) Risk Bounds of Accelerated SGD for Overparameterized Linear Regression and Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP have been accepted to ICLR 2024!
(2024/01) Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models has been released on arXiv 🚀
(2023/11) Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves has been released on arXiv 🚀
(2023/10) I received the NeurIPS 2023 Scholar Award!
(2023/09) Robust Learning with Progressive Data Expansion against Spurious Correlation has been accepted to NeurIPS 2023!
Education
Ph.D. (Computer Science), University of California, Los Angeles, 2021-2026 (expected)
M.Sc. (Computer Science), University of California, Los Angeles, 2019-2021
B.Sc. (Mathematics of Computation), University of California, Los Angeles, 2015-2019
Preprints
Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance.
Linxi Zhao*, Yihe Deng*, Weitong Zhang, Quanquan Gu
arXiv preprint arXiv:2402.08680, Preprints. (pdf)
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models.
Zixiang Chen*, Yihe Deng*, Huizhuo Yuan*, Kaixuan Ji, Quanquan Gu
arXiv preprint arXiv:2401.01335, Preprints. (pdf) (project page)
Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves.
Yihe Deng, Weitong Zhang, Zixiang Chen and Quanquan Gu
arXiv preprint arXiv:2311.04205, Preprints. (pdf) (project page)
PhyGCN: Pre-trained Hypergraph Convolutional Neural Networks with Self-supervised Learning.
Yihe Deng*, Ruochi Zhang*, Pan Xu, Jian Ma and Quanquan Gu
bioaRxiv preprint. (pdf)
2024
Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP.
Zixiang Chen*, Yihe Deng*, Yuanzhi Li, and Quanquan Gu
Proceedings of the Twelfth International Conference on Learning Representations (ICLR), 2024. (pdf)
Risk Bounds of Accelerated SGD for Overparameterized Linear Regression
Xuheng Li, Yihe Deng, Jingfeng Wu, Dongruo Zhou, Quanquan Gu
Proceedings of the Twelfth International Conference on Learning Representations (ICLR), 2024. (pdf)
2023
Robust Learning with Progressive Data Expansion against Spurious Correlation.
Yihe Deng*, Yu Yang*, Baharan Mirzasoleiman, and Quanquan Gu
Advances in Neural Information Processing Systems (NeurIPS), 2023. (pdf) (project page)
2022
Towards Understanding Mixture of Experts in Deep Learning.
Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, Yuanzhi Li
Advances in Neural Information Processing Systems (NeurIPS), 2022. (pdf)
Before Ph.D.
Fast Gradient Projection Method for Text Adversary Generation and Adversarial Training.
Xiaosen Wang*, Yichen Yang*, Yihe Deng*, Kun He.
Association for the Advancement of Artificial Intelligence (AAAI) 2021 (pdf)
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency.
Shuhuai Ren, Yihe Deng, Kun He, Wangxiang Che.
Conference of the Association for Computational Linguistics (ACL) 2019 (oral) (pdf)
Adaptive Wavelet Clustering for High Noise Data.
Zengjian Chen, Jiayi Liu, Yihe Deng, Kun He, John E. Hopcroft.
IEEE International Conference on Data Engineering (ICDE) 2019 (oral) (pdf)
Research Experiences
(2023) Applies Scientist Intern at Amazon AWS, supervised by Dr. Jun Huan.
(2018-2019) Student Researcher at UCLANLP lab, supervised by Prof. Kai-Wei Chang.
(2017-2018) Student Researcher at Department of Math, UCLA, supervised by Prof. Wotao Yin.
Invited Talks
Self-play fine-tuning converts weak language models to strong language models
(2024/03) Invited Talk, UCLA-NLP Group, University of California, Los Angeles
Recent research: Rephrase and Respond & Self-Play Fine-Tuning
(2024/02) Invited Talk, Visual Informatics Group (VITA), University of Texas at Austin
Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves.
(2023/11) Invited talk at Beijing Academy of Artificial Intelligence (BAAI)
(2023/11) AI-Lab NLP Tech Seminar, ByteDance
Teaching Experiences
Teaching Assistant at Department of Computer Science, UCLA
COMSCI 32: Introduction to Computer Science II (Spring 2024)
COMSCI 31: Introduction to Computer Science I (Winter 2024)
COMSCI CM122: Algorithms in Bioinformatics (Spring 2023)
COMSCI M148: Introduction to Data Science (Winter 2023)
COMSCI M146: Introduction to Machine Learning (Fall 2022)
Graduate Grader at Department of Computer Science, UCLA
COMSCI 263: Natural Language Processing (Spring 2020).
Grader at Department of Mathematics, UCLA 2017-2019
MATH 3C: Ordinary Differential Equations with Linear Algebra for Life Sciences Students; MATH 32B: Calculus of Several Variables; and MATH 61: Introduction to Discrete Structures.
Professional Services
Conference Reviewer
Reviewed for NeurIPS (2021-2023), ICML (2023-2024), ICLR (2023-2024), AISTATS (2023-2024), COLM (2024), AAAI (2021-2022), Workshop on Spurious Correlations, Invariance and Stability (2023).
Journal Reviewer
Reviewed for Artificial Intelligence (AI).
Awards
NeurIPS 2023 Scholar Award
UCLA Graduate Division Fellowships, 2022
UCLA Graduate Division Fellowships, 2021
Academic Activities
Participant in CIFAR Deep Learning + Reinforcement Learning Summer School 08/2020