AVRAJIT GHOSH
Embedded Files
I am a postdoc fellow at Simons Institute for the Theory of Computing and BAIR (UC Berkeley EECS). I am affiliated with the Machine Learning Research Pod where I am fortunate to be advised by Peter Bartlett and Bin Yu.
I graduated from Computational Mathematics Science and Engineering department at Michigan State University where I was fortunate to be advised by Rongrong Wang and Saiprasad Ravishankar.
I work on optimization for deep learning with prime focus in:
a) Implicit bias of optimizer/ hyperparameter, b) Statistical benefits/limitations of implicit regularization.
Research Publications: (* denotes equal contribution)
Training dynamics in deep learning: Role of Hyperparameters
1) Variational Learning Finds Flatter Solutions at the Edge of Stability [Learning-rate + weight perturbation]
1) Variational Learning Finds Flatter Solutions at the Edge of Stability [Learning-rate + weight perturbation]
Avrajit Ghosh, Bai Cong, Rio Yokota, Saiprasad Ravishankar, Rongrong Wang, Molei Tao, Mohammad Emtiyaz Khan, Thomas Möllenhoff
(NeurIPS-2025, Spotlight, Top 3.1%)
Avrajit Ghosh*, Soo Min Kwon*, Rongrong Wang, Saiprasad Ravishankar, Qing Qu
(ICLR-2025)
3) Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent. [Learning-rate + momentum]
Avrajit Ghosh*, He Lyu*, Xitong Zhang, Rongrong Wang
(ICLR-2023, Spotlight, Top 5%)
Generalization theory in deep learning
1) PAC-Bayes Generalization bounds for Score Based Diffusion Models
Avrajit Ghosh, Rongrong Wang
(NeurIPS 2025, DynaFront workshop)
2) Improving Generalization of Complex Models with Unbounded Loss Using PAC-Bayes Bounds.
Xitong Zhang, Avrajit Ghosh, Guangliang Wang, Rongrong Wang
(TMLR-2024)
Inverse problems/ Compressed sensing
1) Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization. [Architecture]
Avrajit Ghosh, Xitong Zhang, Kenneth Sun, Qing Qu, Saiprasad Ravishankar, Rongrong Wang
(ICML-2024)
2) Learning Sparsity Promoting Regularizers using Bilevel Optimization.
Avrajit Ghosh, Michael Mccann, Madeline Mitchell, Saiprasad Ravishankar
(SIAM Journal on Imaging Sciences - 2024)
3) Understanding Untrained Deep Models for Inverse Problems: Algorithms and Theory.
Avrajit Ghosh*, Ismail Alkhouri*, Evan Bell*, Shijun Liang, Rongrong Wang, Saiprasad Ravishankar
(IEEE SPM Special Issue on the Mathematics of Deep Learning-2025)
4) Bilevel Learning of L1 Regularizers with Closed-Form Gradients.
Avrajit Ghosh, Michael Mccann, Saiprasad Ravishankar
(ICASSP- 2022)
Page updated
Google Sites
Report abuse