News

• [2025/09] (New Paper) My first solo paper "Prior Forgetting and In-Context Overfitting" was accepted at NeurIPS 2025.

• [2025/09] (New Paper) A joint work with Heebin Yoo, Cheongjae Jang, Dong-Sig Han, Jaein Kim, Seunghyeon Lim, Byoung-Tak Zhang, "How Classifier Features Transfer to Downstream: An Asymptotic Analysis in a Two-Layer Model", was accepted at NeurIPS 2025.

• [2025/08] (New Paper) First Paper from our group! Jonghyun Hong's work "Variance Sensitivity Induces Attention Entropy Collapse and Instability in Transformers" was accepted at EMNLP 2025. Congrats Jonghyun!

• [2025/04] I presented the work "Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors" at ICLR 2025 (Singapore).

• [2025/04] I will be hosting the 1st AI Safety Seminar at HYU. 

• [2025/02] I will be serving as an Area Chair at NeurIPS 2025.

• [2025/01] A joint work with Sokbae Lee at Columbia University, "Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors", was accepted at ICLR 2025.

• [2025/01] I ran the 2nd AI Safety (student) workshop at HYU. 

• [2025]

• [2024/12] I will be serving as an Area Chair at ICML 2025.

• [2024/12] My interview on Trustworthy AI was published as an article in Hanyang Journal.

• [2024/08] I ran the 1st AI Safety workshop at HYU. 

• [2024/04] I will be serving as an Area Chair at NeurIPS 2024.

• [2024/04] I joined the organizing committee for the workshop "High-dimensional Learning Dynamics Workshop: The Emergence of Structure and Reasoning" at ICML 2024.

• [2024]

Selected Ongoing Works

names in bold = group members

• ... (Generalization, Optimization) ...
  Hee-Sung Kim, Hyeonseong Kim, SL
  Under Review

• ... (Self-Supervised Learning, Feature Learning, Learning Dynamics) ...
  Juhwan Kim, SL
  Under Review

Publications

names in bold = group members

• [new!] Prior Forgetting and In-Context Overfitting
  SL
  NeurIPS 2025

We investigate the emergence and dynamics of the two modes of in-context learning during pretraining: task recognition and task learning. We show that the model first learns the task learning and the task recognition abilities together in the beginning, but it (a) gradually forgets the task recognition ability to recall the priorly learned tasks and (b) relies more on the given context in the later phase, which we call (a) prior forgetting and (b) in-context overfitting, respectively.

• [new!] How Classifier Features Transfer to Downstream: An Asymptotic Analysis in a Two-Layer Model
  Heebin Yoo, SL, Cheongjae Jang, Dong-Sig Han, Jaein Kim, Seunghyeon Lim, Byoung-Tak Zhang
  NeurIPS 2025

• [new!] Variance Sensitivity Induces Attention Entropy Collapse and Instability in Transformers
  Jonghyun Hong, SL
  EMNLP 2025

We show that attention entropy collapse induces training instability in LLMs pre-training stage and identify the softmax function’s high sensitivity to attention logits variance as a primary factor. Building on this finding, we demonstrate that (i) ReLU-kernel attention, whose near-linear-kernel approximation to softmax renders it insensitive to variance, and (ii) QK-LayerNorm, which controls variance via normalization, both prevent entropy collapse.

• Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors
  SL, Sokbae Lee
  ICLR 2025
  paper/arxiv/poster/slides

We explore prediction risk as well as estimation risk of the ridgeless least squares under more general regression error assumptions, highlighting the benefits of overparameterization in a more realistic setting that allows for clustered or serial dependence. 

• Implicit Jacobian regularization weighted with impurity of probability output
  SL, Jinseong Park, Jaewook Lee
  ICML 2023
  paper/poster/slides

We show that SGD has an implicit regularization effect on the logit-weight Jacobian norm in neural networks. This regularization effect is weighted with the impurity of the probability output, and thus it is active in a certain phase of training. Based on these findings,  we propose a novel optimization method which leads to similar performance as other SOTA sharpness-aware optimization methods such as SAM and ASAM.

• A New Characterization of the Edge of Stability Based on a Sharpness Measure Aware of Batch Gradient Distribution
  SL, Cheongjae Jang
  ICLR 2023
  paper/poster/slides

We provide a clearer characterization of the Edge of Stability and extend it to general mini-batch SGD. Moreover, based on the analysis, we propose a new scaling rule, LSSR (Linear and Saturation Scaling Rule), between learning rate and batch size.

• A Reparametrization-Invariant Sharpness Measure Based on Information Geometry
  Cheongjae Jang, SL, Yung-Kyun Noh, Frank C. Park
  NeurIPS 2022
  paper/poster

We provide a reparametrization-invariant sharpness measure based on information geometry. In particular, experiments confirm that using our measure as a regularizer in neural network training significantly improves performance.

• GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization
  SL, Hoki Kim, Jaewook Lee
  IEEE TPAMI (Feb. 2023) (IF=23.6; 2nd/145 in AI/CS based on the 2022 JIF)
  paper/code
  
  The effect of proxy-gradients-based adversarial attacks on randomized neural networks highly relies on the directional distribution of the loss-input gradients. We propose Gradient Diversity (GradDiv) regularizations that minimize the concentration of the gradients to weaken these proxy-gradients-based attacks.
  
  

• Towards Better Understanding of Training Certifiably Robust Models to Adversarial Examples
  SL, Woojin Lee, Jinseong Park, Jaewook Lee
  NeurIPS 2021
  paper/code/poster/slides
  
  We identify smoothness of the objective loss landscape as an important factor in building certifiably robust model against adversarial attacks.
  
  

• Lipschitz-Certifiable Training with a Tight Outer Bound
  SL, Jaewook Lee, Saerom Park
  NeurIPS 2020
  paper/code/poster/slides
  
  Certifiable training minimizes an upper bound on the worst-case loss over valid adversarial perturbations, and thus the tightness of the upper bound is crucial. We propose a certified defense method with a tight upper bound.

(For more publications, please refer to my CV below)