Zeke Xie's Homepage
"Towards AI Science."
Zeke Xie （谢 泽柯） Assistant Professor of Data Science and Analytics/Artificial Intelligence, HKUST (GZ) Email: zekexie at hkust-gz.edu.cn
My mission is to find a way towards AI Science.
Biography
Dr. Zeke Xie is an Assistant Professor jointly appointed by DSA Thrust and AI Thrust, Hong Kong University of Science and Technology (Guangzhou). I will also be an affiliated Assistant Professor at Hong Kong University of Science and Technology (Clear Water Bay).
Previously, I was a researcher at Baidu Research responsible for large models and AIGC research. I obtained Ph.D. and M.E. both from The University of Tokyo. I was fortunate to be advised by Prof. Issei Sato and Prof. Masashi Sugiyama. I was also affiliated with RIKEN AIP during the Ph.D. study. Before that, I obtained Bachelor of Science from University of Science and Technology of China.
My mission is to find a way towards AI Science (the science of AI). I am generally interested in understanding and solving fundamental issues of modern AI by scientific principles and methodology. My current research projects focus on foundations of machine learning and generative AI.
I wrote a number of popular science articles and tech blogs. I am glad to share my thoughts and communicate with more than 200k followers who are interested in my articles/blogs.
Openings
I am recruiting multiple PhD students, RAs, and postdocs who can enjoy the mission and the experience of exploring the boundary of human knowledge. Students who truly enjoy the process itself and share similar research interests are highly welcomed to work with me.
We will definitely gain both memorable experiences and rich results even beyond the initial expectation.
News
2024.06.28: I have an invited talk at BIMSA, Beijing.
2024.01.16: Our two papers on Neural Field Classifiers and Poisson Learning are accepted at ICLR 2024.
2023.09.22: Our two papers on Weight Decay and Gradient Structure are accepted at NeurIPS 2023.
2023.09.13: I have an invited talk "Deep Learning Dynamics: A Scientific Approach" at DeepSeek/High-Flyer AI.
2023.07.14: Our paper "S3IM: Stochastic Structural SIMilarity and Its Unreasonable Effectiveness for Neural Fields" is accepted at ICCV 2023.
2023.01.21: Our paper "Dataset Pruning: Reducing Training Data by Examining Generalization Influence" is accepted at ICLR 2023.
2023.01.16: I have an invited talk "Deep Learning Dynamics: On Minima Selection and Saddle-Point Escaping" at School of Electrical and Computer Engineering, Peking University.
2022.08.18: I have an invited talk "Deep Learning Dynamics: On Minima Selection and Saddle-Point Escaping" at the First China Conference on Scientific Machine Learning, Peking University.
2022.07.20: I have a long oral "Adaptive Inertia: Disentangling the effects of adaptive learning rate and momentum" at ICML 2022 main conference.
2022.05.15: Our two papers are accepted at ICML 2022 with one oral selection (～2%).
2021.11.07: I have an invited talk "Understanding and Improving Deep Learning Dynamics" at Swarma-Kaifeng Seminar.
2021.10.13: I join Baidu Research as a full-time researcher.
2021.07.19: I successfully defend my Ph.D. thesis and hold the Ph.D. degree.
2021.07.18: My recorded talk "Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization" at ICML 2021 is publicly available.
2021.07.08: I have an invited talk "Understanding and Improving Deep Learning Dynamics" at Noah's Ark Lab, Huawei.
2021.06.29: I have an invited talk "Bridging Deep Learning Dynamics and Computational Neuroscience" at IDG/McGovern Institute for Brain Research, Peking University.
2021.06.25: I have an invited talk "Understanding and Improving Deep Learning Dynamics" at MSRA.
2021.05.13: I give a talk "Deep Learning Dynamics: Diffusion Perspectives" at University of Tokyo.
2021.05.08: Our paper "Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization" is accepted at ICML 2021.
2021.05.06: My recorded talk "A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima" at ICLR 2021 is publicly available.
2021.03.26: Our paper "Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting" is accepted at Neural Computation, MIT Press.
2021.01.13: Our paper "A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima" is accepted at ICLR 2021.
Towards AI Science
Nowadays AI is like the physics in or even before the era of Galileo.
Researchers may observe many interesting things about AI.
However, we have no mathematical theory for most things.
We need to find a road towards the era of Newton for AI.
Science not only explains what works but also predicts what will work.
Science gives quantitative and trustworthy results.
Science establishes complex principles from first principles.
We believe formulating AI Science will be the most important challenge in the future of AI.
We hope to find a road towards the scientific revolution for AI.
This is a mission in our generation.
"All models are wrong. But some are useful."