Advances in machine learning are reshaping the foundations of science and technology. Despite their empirical success, a rigorous theoretical understanding of when and why machine learning algorithms succeed or fail remains limited. My research aims to bridge this gap by developing theoretical frameworks that explain the underlying mechanisms of machine learning and guide the design of principled and effective algorithms. In particular, I aim to understand the limitations of existing methods and to develop theoretically grounded solutions that address these limitations. My current research focuses on language models, representation learning, and synthetic data generation.
Selected Publications and Preprints
For a complete list of my work, see my CV.
Language Models:
How to Correctly Report LLM-as-a-Judge Evaluations [arxiv] [github]
Chungpa Lee, Thomas Zeng, Jongwon Jeong, Jy-yong Sohn, Kangwook Lee
(under review)Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models [arxiv]
Chungpa Lee, Jy-yong Sohn, Kangwook Lee
(under review)
Representation Learning:
On the Similarities of Embeddings in Contrastive Learning [paper] [arxiv] [github]
Chungpa Lee, Sehee Lim, Kibok Lee, Jy-yong Sohn
In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning [paper] [arxiv] [github]
Chungpa Lee, Jeongheon Oh, Kibok Lee, Jy-yong Sohn
In Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS), 2025Analysis of Using Sigmoid Loss for Contrastive Learning [paper] [arxiv] [github]
Chungpa Lee, Joonhwan Chang, Jy-yong Sohn
In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), 2024
Synthetic Data Generation: