Berivan Isik

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities [PDF]

Gemini team, 2025

Publications

• Highlights:

The Magic Correlations: Understanding Knowledge Transfer from Pretraining to Supervised Fine-Tuning [PDF]

Simin Fan, Dimitris Paparas, Natasha Noy, Binbin Xiong, Noveen Sachdeva, Berivan Isik

Preprint, 2026.

Leveraging Per-Instance Privacy for Machine Unlearning [PDF]

Naz Sepahvand, Anvith Thudi, Berivan Isik, Ashmita Bhattacharyya, Nicolas Papernot, Eleni Triantafillou, Daniel Roy, Gintare Karolina Dziugaite

International Conference on Machine Learning (ICML), 2025.

Also presented at the Theory and Practice of Differential Privacy (TPDP) Workshop in 2025.  [Oral]

Scaling Laws for Downstream Task Performance of Large Language Models [PDF]

Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, Sanmi Koyejo

International Conference on Learning Representations (ICLR), 2025.

Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs [PDF] [code]

Ashwinee Panda, Berivan Isik, Xiangyu Qi, Sanmi Koyejo, Tsachy Weissman, Prateek Mittal

Preprint, 2024.

Preliminary version presented at the Workshop on Advancing Neural Network Training (WANT): Computational Efficiency, Scalability, and Resource Optimization, ICML, 2024. [Best Paper Award, Oral]

Preliminary version presented at the Workshop on Efficient Systems for Foundation Models (ES-FoMo), ICML, 2024. [Oral]

Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation [PDF] [code]

Berivan Isik, Wei-Ning Chen, Ayfer Ozgur, Tsachy Weissman, Albert No

Conference on Neural Information Processing Systems (NeurIPS), 2023.

Preliminary version presented at the Theory and Practice of Differential Privacy (TPDP) Workshop in 2023.

Sparse Random Networks for Communication-Efficient Federated Learning  [PDF] [code] 

Berivan Isik*, Francesco Pase*, Deniz Gunduz, Tsachy Weissman, Michele Zorzi 

International Conference on Learning Representations (ICLR), 2023.

Preliminary version presented at the Workshop on Federated Learning, NeurIPS, 2022. [Oral]

An Information-Theoretic Justification for Model Pruning [PDF] [code] 

Berivan Isik, Tsachy Weissman, Albert No

International Conference on Artificial Intelligence and Statistics (AISTATS), 2022.

Preliminary version presented at the Sparsity in Neural Networks Workshop, 2021. [Spotlight talk]

• Others:

Differentially Private Adaptation of Diffusion Models via Noisy Aggregated Embeddings [PDF]

Pura Peetathawatchai, Wei-Ning Chen, Berivan Isik, Sanmi Koyejo, Albert No

IEEE Secure and Trustworthy Machine Learning Conference (SaTML), 2026.

Sketching for Distributed Deep Learning: A Sharper Analysis [PDF]

Mayank Shrivastava, Berivan Isik, Qiabo Li, Arindam Banerjee,  Sanmi Koyejo

Conference on Neural Information Processing Systems (NeurIPS), 2024.

Improved Communication-Privacy Trade-offs in L2 Mean Estimation under Streaming Differential Privacy [PDF]

Wei-Ning Chen, Berivan Isik, Peter Kairouz, Albert No, Sewoong Oh, Zheng Xu

International Conference on Machine Learning (ICML), 2024.

On Fairness of Low-Rank Adaptation of Large Models [PDF] [code]

Zhoujie Ding, Ken Liu, Pura Peetathawatchai, Berivan Isik, Sanmi Koyejo

Conference on Language Modeling (COLM), 2024.

Adaptive Compression in Federated Learning via Side Information [PDF] [code]

Berivan Isik*, Francesco Pase*, Deniz Gunduz, Sanmi Koyejo, Tsachy Weissman, Michele Zorzi 

International Conference on Artificial Intelligence and Statistics (AISTATS), 2024.

GPT-Zip: Deep Compression of Finetuned Large Language Models [PDF]

Berivan Isik*,  Hermann Kumbong*, Wanyi Ning*, Xiaozhe Yao*, Sanmi Koyejo, Ce Zhang

Preliminary version presented at the Workshop on Efficient Systems for Foundation Models, ICML, 2023.

Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers [PDF][code]

Onur Guleryuz, Philip A. Chou, Berivan Isik, Hugues Hoppe, Danhang Tang, Ruofei Du, Jonathan Taylor, Philip Davidson, Sean Fanello

Submitted to the IEEE Transactions on Image Processing (TIP), 2024.

Mutual Information Upper Bounds for Uniform Inputs through the Deletion Channel [PDF]

Francisco Pernice, Berivan Isik, Tsachy Weissman

IEEE Transactions on Information Theory (TIT), 2024.

Lossy Compression of Noisy Data for Private and Data-Efficient Learning [PDF]

Berivan Isik, Tsachy Weissman

IEEE Journal of Selected Areas in Information Theory (JSAIT), 2023.

Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers [PDF] [code]

Berivan Isik, Onur Guleryuz, Danhang Tang, Jonathan Taylor, Philip A. Chou

IEEE International Conference on Image Processing (ICIP), 2023.

Neural Network Compression for Noisy Storage Devices [PDF]

Berivan Isik, Kristy Choi, Xin Zheng, Tsachy Weissman, Stefano Ermon, H.-S. Philip Wong, Armin Alaghi

ACM Transactions on Embedded Computing Systems (TECS), 2023.

Learning under Storage and Privacy Constraints [PDF1] [PDF2 (longer)]

Berivan Isik, Tsachy Weissman

IEEE International Symposium on Information Theory (ISIT), 2022.

LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks [PDF] [code] 

Berivan Isik, Philip A. Chou, Sung Jin Hwang, Nick Johnston, George Toderici

Journal of Frontiers in Signal Processing (FSP), 2022.

Neural 3D Compression via Model Compression  [PDF]

Berivan Isik

WiCV, CVPR, 2021.

rTop-k: A Statistical Estimation Approach to Distributed SGD [PDF] 

Berivan Isik*,  Leighton P. Barnes*,  Huseyin A. Inan*,  Ayfer Ozgur

IEEE Journal on Selected Areas in Information Theory (JSAIT), 2020. 

Preliminary version presented at the Federated Learning for User Privacy and Data Confidentiality, ICML, 2020. [Long Oral]

PhD Thesis

Statistical methods for efficient and trustworthy machine learning (2024) 

Patents

 "Learned Volumetric Attribute Compression Using Coordinate-Based Networks", US Patent 17/708,628.

Philip A. Chou, Berivan Isik, Sung Jin Hwang, Nick Johnston, George Toderici 

Invited Talks

1. Artificial Intelligence Summer School, Istanbul. "Evaluating Large Language Models: Methods and Challenges", July 2026.

2. Stanford-Berkeley Research Meetup. April 2026.

3. Princeton University - Alg-ML Seminar. "Beyond Pretraining Loss: Predicting Downstream Performance of LLMs", November 2025. 

4. ICCV Tutorial. "Learning Deep Low-Dimensional Models from High-Dimensional Data: From Theory to Practice", Oct. 2025.

5. DEMi Summit. "Beyond Pretraining Loss: Predicting Downstream Performance of LLMs", June 2025. 

6. Middle East Technical University - AI Society. "After METU: AI Research at Stanford and Google", April 2025.

7. UC Berkeley -  Berkeley Laboratory for Information and System Sciences (BLISS) Seminar. "Beyond Pretraining Loss: Evaluating Value of Pretraining Data for Large Language Models at Scale", April 2025.

8. Middle East Technical University - Guest Lecture. "Beyond Pretraining Loss: Evaluating Value of Pretraining Data for Large Language Models at Scale", April 2025.

9. Bilkent University - Guest Lecture. "Beyond Pretraining Loss: Evaluating Value of Pretraining Data for Large Language Models at Scale", Feb. 2025.

10. University of Southern California - AI Foundations Seminar. "Value of Pretraining Data: Scaling Laws for Downstream Task Performance of Large Language Models", Oct. 2024.

11. Vector Institute for Artificial Intelligence. "Scaling Laws for Downstream Task Performance of Large Language Models", March 2024.

12. INFORMS Optimization Society (IOS) Conference - Rice University. "Sparse Random Networks for Communication-Efficient Federated Learning", March 2024. 

13. Columbia University - NYC Privacy Day. "Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation", Dec. 2023.

14. Meta. "Federated Learning under Communication and Privacy Constraints", Oct. 2023. 

15. Google - Federated Learning Seminar. "Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation", Oct. 2023. 

16. Vector Institute for Artificial Intelligence. "Sparse Random Networks for Communication-Efficient Federated Learning", Sep. 2023.

17. Video Quality Expert Group (VQEG). "Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers", June 2023.

18. Flower Monthly. "Sparse Random Networks for Communication-Efficient Federated Learning", June 2023.

19. Federated Learning One World (FLOW) Seminar. "Sparse Random Networks for Communication-Efficient Federated Learning", March 2023. [Slides]

20. Google - Federated Learning Seminar. "Sparse Random Networks for Communication-Efficient Federated Learning", Jan. 2023. [Slides]

21. Bilkent University - Electrical Engineering Seminars. "Sparsity in Neural Networks", Jan. 2023.  [Slides]

22. Middle East Technical University - Electrical Engineering Seminars. "Sparsity in Neural Networks", Dec. 2022. [Slides]

23. Google - Sparsity Reading Group. "An Information-Theoretic Justification for Model Pruning", Nov. 2022. 

24. Google - Perception. "Learned Sandwiched Video Compression", Oct. 2022.

25. Google - Information Theory Reading Group. "An Information-Theoretic Justification for Model Pruning", Sep. 2022. 

26. Google - Information Theory Reading Group. "LCoN: Learning Under Storage and Privacy Constraints", July 2022.

27. Google - Neural Compression Team. "LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks", Sep. 2021.

28. Stanford Compression Workshop. "An Information-Theoretic Approach to Neural Network Compression", Feb. 2021. [Video] [Slides]

29. NeurIPS Meetup Turkey. "Noisy Neural Network Compression", Dec. 2020. [Video] (Turkish)

30. Facebook Reality Labs - Research. "Robust Neural Network Compression", Nov. 2020.

Involvements

• Area Chair for NeurIPS 2026. 

• Reviewer for ICML, NeurIPS, ICLR, AISTATS, AAAI, COLM, ECCV, CVPR, ICCV, JMLR, IJCAI, ACM FAccT, DMLR, SaTML, SIMODS, IEEE JSAIT, IEEE ISIT, IEEE ITW, IEEE TPAMI, Data Compression Conference, until 2026. 

• Member of the Stanford Faculty Search Committee, 2024. 

• Lead organizer of the NeurIPS 2025 Workshop -- Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling. [workshop webpage]

• Organizer of the ICML 2025 Workshop: "Tiny Titans: The next wave of On-Device Learning for Foundational Models" [workshop webpage]

• Organizer of the ICLR 2025 Workshop on Sparsity in LLMs (SLLM): Deep Dive into Mixture of Experts, Quantization, Hardware, and Inference. [workshop webpage]

• Lead organizer of the ICML 2024 Workshop on Theoretical Foundations of Foundation Models (TF2M). [workshop webpage]

• Organizer of the ICML 2024 Workshop on Data-centric Machine Learning Research (DMLR): Datasets for Foundation Models. [workshop webpage]

• Lead organizer of the ICML 2023 Workshop on Neural Compression: From Information Theory to Applications. [workshop webpage]

• Organizer of the Women in Machine Learning (WiML) Workshop at ICML 2021.

• Organizer of the ICML 2021 Workshop on Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine Learning (ITR3@ICML-21). [workshop webpage]

• Member of the ICML 2024, NeurIPS 2024, ICLR 2025, ICML 2025, and NeurIPS 2025 Workshop Proposal Committee.

• Reviewer for the ICLR Blogposts track, 2023-2024.

• Mentor for STEM to SHTEM summer internship program for high school students in 2020. 

Hobbies