Short Bio [CV]
Cheolhyoung Lee is a post-doctoral associate of Center for Data Science at New York University under the supervision of Kyunghyun Cho. He received his Ph.D . and M.Sc. degrees from Department of Mathematical Sciences at Korea Advanced Institute of Science and Technology (KAIST) under the supervision of Wanmo Kang. He has been studying probability and optimization with emphasis on their applications to machine learning. He is currently interested in explaining the dynamics of optimizations in deep learning as well as other mathematical aspects in deep learning with particular focus on natural language processing.
Working Experience
Post-doctoral Associate of Center for Data Science at New York University, Sep. 2020 - Present
Academic Background
Ph.D., Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Aug. 2020
Thesis: Design and Analysis of Optimization Problems in Deep Learning
M.Sc., Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Aug. 2014
Thesis: The Marginal Distributions of Multi-tail Generalized Elliptical Distributions
B.Sc., Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Feb. 2012
Contact Information
cheolhyoung.lee [at] nyu [dot] edu
Publications
Unsupervised Learning of Initialization in Deep Neural Networks via Maximum Mean Discrepancy (Cheolhyoung Lee and Kyunghyun Cho), Preprint [pdf]
A Non-monotonic Self-terminating Language Model (Eugene Choi, Kyunghyun Cho, and Cheolhyoung Lee), 11th International Conference on Learning Representations (2023) [pdf]
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models (Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang), 8th International Conference on Learning Representations (2020) [pdf]
Directional Analysis of Stochastic Gradient Descent via von Mises-Fisher Distributions in Deep learning (Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang), “Integration of Deep Learning Theories” Workshop, 32nd Conference on Neural Information Processing Systems (2018) [pdf]
Honors and Awards
1st prize at SNUH Medical AI Challenge 2020 (Intraoperative hypotension prediction), Nov. 2020
2nd prize at Korea Health Datathon 2020 (Sinusitis classification from X-ray images), Oct. 2020
2nd prize at Digital Health Hackathon 2020 (Treatment decision from survival time prediction), Sep. 2020
KAIST-Google Partnership Program (Student Travel Grant), Feb. 2020
2nd prize at AI HeLP Challenge 2019 (Critical event prediction at NICU), Feb. 2020