Cheolhyoung Lee

Short Bio [CV]

Cheolhyoung Lee is a post-doctoral associate of Center for Data Science at New York University under the supervision of Kyunghyun Cho.  He received his Ph.D . and M.Sc. degrees from Department of Mathematical Sciences at Korea Advanced Institute of Science and Technology (KAIST) under the supervision of Wanmo Kang. He has been studying probability and optimization with emphasis on their applications to machine learning. He is currently interested in explaining the dynamics of optimizations in deep learning as well as other mathematical aspects in deep learning with particular focus on natural language processing. 

Working Experience

• • Post-doctoral Associate of Center for Data Science at New York University, Sep. 2020 - Present

Academic Background

• • Ph.D., Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Aug. 2020

    • Thesis: Design and Analysis of Optimization Problems in Deep Learning

  • M.Sc., Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Aug. 2014

    • Thesis: The Marginal Distributions of Multi-tail Generalized Elliptical Distributions

  • B.Sc., Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Feb. 2012

Contact Information

• • cheolhyoung.lee [at] nyu [dot] edu

Publications

• 1. Unsupervised Learning of Initialization in Deep Neural Networks via Maximum Mean Discrepancy (Cheolhyoung Lee and Kyunghyun Cho), Preprint [pdf]

  2. A Non-monotonic Self-terminating Language Model (Eugene Choi, Kyunghyun Cho, and Cheolhyoung Lee), 11th International Conference on Learning Representations (2023)  [pdf] 

  3. Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models (Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang), 8th International Conference on Learning Representations (2020)  [pdf]

  4. Directional Analysis of Stochastic Gradient Descent via von Mises-Fisher Distributions in Deep learning (Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang), “Integration of Deep Learning Theories” Workshop, 32nd Conference on Neural Information Processing Systems (2018)  [pdf]

Honors and Awards

• • 1st prize at SNUH Medical AI Challenge 2020 (Intraoperative  hypotension prediction), Nov. 2020

  • 2nd prize at Korea Health Datathon 2020 (Sinusitis classification from X-ray images), Oct. 2020

  • 2nd prize at Digital Health Hackathon 2020 (Treatment decision from survival time prediction), Sep. 2020

  • KAIST-Google Partnership Program (Student Travel Grant), Feb. 2020

  • 2nd prize at AI HeLP Challenge 2019 (Critical event prediction at NICU), Feb. 2020