Fall 2022 (Sep - Nov)

[11/28/2022] Narrow NTK & Acceleration in Manifold Optimization

Neural Tangent Kernel Analysis of Deep Narrow Neural Networks (Lee et al., ICML 2022)

Presenter (offline): Jongmin Lee (SNU Math)
Slide and Paper

Accelerated Gradient Methods for Geodesically Convex Optimization: Tractable Algorithms and Convergence Analysis (Kim & Yang, ICML 2022 Spotlight)

Presenter (online): Jungbin Kim (SNU Math&IE Undergrad)
Slide and Paper

[11/21/2022] Acceleration in Optimization Theory

Continuous-Time Analysis of Accelerated Gradient Methods via Conservation Laws in Dilated Coordinate Systems (Suh et al., ICML 2022 long presentation)

Presenter (offline): Jaewook Suh (SNU Math)
Slide and Paper

Exact Optimal Accelerated Complexity for Fixed-Point Iterations (Park & Ryu, ICML 2022 long presentation)

Presenter (offline): Jisun Park (SNU Math)
Slide and Paper

[11/14/2022] Intriguing Properties of GD & PINN

Properties of Gradient Descent on Neural Networks

Presenter (offline): Hojung Jung (OSI Lab @KAIST AI)
Slide
Papers presented:
- Implicit Gradient Regularization (Barrett et al., ICLR 2021)
- Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability (Cohen et al., ICLR 2021)
- Stochastic Training is Not Necessary for Generalization (Geiping et al., ICLR 2022)

Constants of Motion Network (Kasim & Lim., NeurIPS 2022)

Presenter (online): Yeongwoo Song (CSSPL @KAIST Physics)
Slide and Paper

[10/31/2022] Loss Landscapes of Neural Networks and Spin Glass - Part 2

The Loss Surfaces of Multilayer Networks (Choromanska et al., AISTATS 2015)

Presenter (offline): Minchan Jeong (OSI Lab @KAIST AI)
Paper (+ Open Problem (COLT 2015))

[10/24/2022] Loss Landscapes of Neural Networks and Spin Glass - Part 1

Comparing Dynamics: Deep Neural Networks versus Glassy Systems (Baity-Jesi et al., ICML 2018; Journal of Statistical Mechanics 2019)

Presenter (offline): Hanseul Cho (OptiML Lab @KAIST AI)
Slide and Paper

[10/17/2022] Recent Works by Chizat and Bach on GD for Two-Layer Networks

Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization (Bach & Chizat, ICM 2022)

Presenter (offline): Junghyun Lee (OSI Lab & OptiML Lab @KAIST AI)
Slide and Paper

[09/26/2022] SGD & Loss Landscapes

Strength of Minibatch Noise in SGD (Ziyin et al., ICLR 2022 Spotlight)

Presenter (online): Yujeong Lee (Samsung Research)
Slide and Paper

Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry (Pittorino et al., ICML 2022)

Presenter (offline): Hojung Jung (OSI Lab @KAIST AI)
Slide and Paper

[09/19/2022] Local Behaviors of SGD

Behavior of SGD around local minimum

Presenter (offline): Hojung Jung (OSI Lab @KAIST AI)
Slide
Papers presented:
- Three Factors Influencing Minima in SGD (Jastrzebski et al., ICANN 2018)
- Power-Law Escape Rate of SGD (Mori et al., ICML 2022)

SGD with a Constant Large Learning Rate Can Converge to Local Maxima (Ziyin et al., ICLR 2022 Spotlight)

Presenter (online): Yujeong Lee (Samsung Research)
Slide and Paper

[09/05/2022] PINN & Loss Landscape Through the Lens of Statistical Mechanics

Meta-learning Hamilton's equation

Presenter (online): Yeongwoo Song (CSSPL @KAIST Physics)
Slide

Landscape and training regimes in deep learning (Geiger et al., Physics Report 2021)

Presenter (online): Junghyun Lee (OSI Lab & OptiML @KAIST AI)
Slide and Paper

Page updated

Google Sites

Report abuse