Surrogate Gap Guided Minimiation Improves Sharpness-Aware Training
ICLR 2022
Juntang Zhuang [1], Boqing Gong [2], Liangzhe Yuan [2], Yin Cui [2], Hartwig Adam [2], Nicha C. Dvornek [1], Sekhar Tatikonda [1], James S. Duncan [1], Ting Liu [2]
[1] Yale University
[2] Google Research
Lower curvature leads to better generalization
Comparison of different training schemes
Validations on toy examples and deep-learning experiments
Citation
@inproceedings{
zhuang2022surrogate,
title={Surrogate Gap Minimization Improves Sharpness-Aware Training},
author={Juntang Zhuang and Boqing Gong and Liangzhe Yuan and Yin Cui and Hartwig Adam and Nicha C Dvornek and sekhar tatikonda and James s Duncan and Ting Liu},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=edONMAnhLu-}
}