Research Interest

My research interests include optimization algorithms, algorithmic efficiency, differential privacy, deep learning theory, distributed learning, and high-dimensional statistics.

Publication

1. Differentially Private Optimization on Large Model at Small Cost

Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, George Karypis

ICML, 2023

Available on AWS github, 2022 [code] (7,1,6)

2. Zero redundancy distributed learning with differential privacy

Zhiqi Bu, Justin Chiu, Ruixuan Liu, Yu-Xiang Wang, Sheng Zha, George Karypis

ICLR 2023 Workshop Trustworthy ML (Oral), 2023 (8,0,s11/23)

3. Differentially Private Bias-Term only Fine-tuning of Foundation Models

Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, George Karypis

NeurIPS TSRML workshop (Best paper award), 2022 (3,1,s11/22)

4. Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger

Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, George Karypis

NeurIPS, 2023 (4,2,12)

ICML TPDP workshop, 2022

5. Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach

Xinwei Zhang, Zhiqi Bu, Zhiwei Steven Wu, Mingyi Hong

6. Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy

Zhiqi Bu*, Jialin Mao*, Shiyun Xu

NeurIPS, 2022 [code] (3,1,4)

7. Differentially Private Optimizers Can Learn Adversarially Robust Models

Zhiqi Bu*, Yuan Zhang*

ICML TPDP workshop, 2022

TMLR 2023 (4,2, 12)

8. CEDAR: Communication Efficient Distributed Analysis for Regressions

Changgee Chang, Zhiqi Bu, Qi Long

Biometrics, 2022

9. Asymptotic Statistical Analysis of Sparse Group LASSO via Approximate Message Passing

Kan Chen, Zhiqi Bu, Shiyun Xu

NeurIPS, Optimization for Machine Learning (OPT) workshop, 2020

ECML 2021 (4,2,17)

10. "On the Convergence and Calibration of Deep Learning with Differential Privacy"

Zhiqi Bu, Hua Wang, Zongyu Dai, Qi Long

ICML, Theory and Practice of Differential Privacy workshop 

TMLR 2023 [slides][code] (3,1, 24)

11. "MISNN: Multiple Imputation via Semi-parametric Neural Networks"

Zhiqi Bu*, Zongyu Dai*, Yiliang Zhang*, Qi Long

ICML, Subset Selection in Machine Learning workshop (spotlight) 2021 [slides][code] (1,1,5)

12. "Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability"

Zhiqi Bu*, Qiyiwen Zhang*, Kan Chen, Qi Long

ICML, Theory and Practice of Differential Privacy workshop 2021 [slides][code] (3,1,8)

ECML 2022

13. "Multiple Imputation via Generative Adversarial Network for High-dimensional Blockwise Missing Value Problems"

Zongyu Dai, Zhiqi Bu, Qi Long

IEEE International Conference on Machine Learning and Applications, 2021 [slides][code] (12,2,6)

14. "Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity"

Zhiqi Bu*, Shiyun Xu*, Pratik Chaudhari, Ian Barnett

ICLR, Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data workshop, 2022

ECML 2023 [slides][code] (10,2,6)

15. "Fast and Memory Efficient Differentially Private-SGD via JL Projections"

Zhiqi Bu, Sivakanth Gopi, Janardhan Kulkarni, Yin Tat Lee, Judy Hanwen Shen, Uthaipon Tantipongpipat

NeurIPS, 2021 [slides][code] (3,1,9)

16. "Accuracy, Interpretability, and Differential Privacy via Explainable Boosting"

Harsha Nori, Zhiqi Bu, Judy Hanwen Shen, Rich Caruana, Janardhan Kulkarni

ICML (spotlight) 2021 [slides][code] (2,1,6)

17. "Privacy Amplification via Iteration for Shuffled and Online PNSGD"

Matteo Sordello, Zhiqi Bu, Jinshuo Dong

ICLR, Distributed and Private Machine Learning workshop (oral) 2021

ECML 2021 [slides][code] (1,1,4)

18. "Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit"

Zhiqi Bu, Jason Klusowski, Cynthia Rush, Weijie Su

Annals of Statistics [slides][code] (20,4,12)

19. "A dynamical view on optimization algorithms of overparameterized neural networks"

Zhiqi Bu, Shiyun Xu, Kan Chen

NeurIPS, Optimization for Machine Learning (OPT) workshop, 2020

AISTATS 2021 [slides][code] (1,1, 4)

20. “Efficient Designs of SLOPE Penalty Sequences in Finite Dimension”

Zhiqi Bu*, Yiliang Zhang*

NeurIPS, Optimization for Machine Learning (OPT) workshop, 2020

AISTATS 2021 [slides][code] (1,1,4)

21. “DebiNet: Debiasing Linear Model with Over-parameterized Neural Networks”

Zhiqi Bu*, Shiyun Xu*

AISTATS 2021 [slides][code] (3,2,6)

22. "Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passing"

Zhiqi Bu, Jason Klusowski, Cynthia Rush, Weijie Su

IEEE Transactions on Information Theory 2020 [slides][code] (7,2,13)

23. "The Complete Lasso tradeoff diagram"

Hua Wang, Yachong Yang, Zhiqi Bu, and Weijie Su

NeurIPS (spotlight) 2020 [slides][code] (1,1,4)

24. "Deep Learning with Gaussian Differential Privacy"

Zhiqi Bu, Jinshuo Dong, Qi Long, Weijie J. Su

Harvard Data Science Review 2019 [slides][code(Pytorch)][code(Tensorflow)] (3,1,7)

25. "Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passing"

Zhiqi Bu, Jason Klusowski, Cynthia Rush, Weijie Su

NeurIPS 2019 [slides][code] (7,2,4)

* indicates equal contribution

Talks and Posters

• WHOA-PSI, Washington University in St. Louis, 2019, "Estimation and Inference of SLOPE via Approximate Message Passing"

• NIPS, Vancouver, Canada, 2019, "Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passing"

• ICSA International Conference, Zhejiang University, China, 2019, "SLOPE meets AMP:Does SLOPE outperform LASSO?"

• Mathematical Methods of Modern Statistics 2, Marseille, France, 2020, "SLOPE meets AMP:Does SLOPE outperform LASSO?"

• Joint Statistical Meetings (JSM), Philadelphia, 2020, "Deep Learning with Gaussian Differential Privacy"

• Microsoft Research Seminar, 2020, “Fast Differentially Private Deep Learning with Projection”

• LinkedIn Maching Learning reading group, 2020, "Differentially Private Deep Learning Review"

• NeurIPS, Vancouver, 2020, “A dynamical view on optimization algorithms of overparameterized neural networks”

• AISTATS, 2021, "A dynamical view on optimization algorithms of overparameterized neural networks"

• Joint Statistical Meeting, 2021, "A dynamical view on optimization algorithms of overparameterized neural networks"

• Baidu Research, Seattle, 2021 "On the Convergence of Deep Learning with Differential Privacy", 

• ICLR 2021, "Privacy Amplification via Iteration for Shuffled and Online PNSGD"

• ICML 2021, "Accuracy, Interpretability and Differential Privacy via Explainable Boosting"

• ICML 2021, "Missing Value Imputation with Semiparametric Neural Network"

• Statistical Learning Seminars, Wrocław University, 2021, "Characterizing the SLOPE Trade-Off: A Variational perspective and the Donoho--Tanner Limit"

• Google Research privacy seminar, 2021, "On the Convergence of Deep Learning with Differential Privacy"

• University of California, Santa Barbara, 2021, "On the Convergence of Deep Learning with Differential Privacy"

• IBM research seminar, 2021, "On the Convergence and Calibration of Deep Learning with Differential Privacy"

• Baidu Research, Seattle, 2021 "Practical Adversarial Training with Differential Privacy" 

• Amazon AWS AI, New York, 2021, "Large-scale Deep Learning with Differential Privacy"

• Bell Lab, New Jersey, 2021, "A dynamical view on optimization algorithms of overparameterized neural networks"

• Alibaba Ant Group, 2021, "On the Convergence and Calibration of Deep Learning with Differential Privacy"

• ICML, Baltimore, 2022, "To be private and robust: Differentially Private Optimizers Can Learn Adversarially Robust Models"

• ICML, Baltimore, 2022, "Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger”

• NeurIPS, New Orleans, 2022, "Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy"

• NeurIPS, New Orleans, 2022, "Differentially Private Bias-Term only Fine-tuning of Foundation Models"

• ICLR, 2023, "Zero redundancy distributed learning with differential privacy”

• ICML, Hawaii, 2023, "Differentially Private Optimization on Large Model at Small Cost"

• University of Minnesota, 2023, "On the Computational Efficiency of Differentially Private Deep Learning"

• NeurIPS, New Orleans, 2023, "Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger"

Academic Service 

• 2020: IMAIAI

• 2021: AISTATS, ICML, NeurIPS, IMAIAI

• 2022: ICLR, AAAI, AISTATS, ICML, NeurIPS

• 2023: ICLR, AAAI, AISTATS, ICML, NeurIPS, TMLR

Awards

• NeurIPS Student Travel Award, 2019

• NeurIPS Outstanding Reviewer Award, 2021

• NeurIPS TSRML Best Paper Award, 2022