About

I am a postdoctoral associate at NYU's Center for Data Science working with Joan Bruna. Before that, I was a Ph.D. student at the Weizmann Institute of Science, where I was very fortunate to be advised by Ohad Shamir.  Before that, I did a M.Sc. and B.Sc. in mathematics at Tel Aviv University.  

My main research interests are theoretical machine learning, specifically the theory of deep learning, and related questions about the optimization, generalization, and expressivity of neural networks.  I also enjoy working on practical questions related to deep learning or optimization which combine theoretical insights. Recently I have also been interested in the theory behind generative models, specifically transformers and attention mechanisms.

I interned at Google Israel, where I worked on large-scale optimization problems, and at NVIDIA Research where I worked with Haggai Maron and Gal Chechik on graph neural networks. Before my Ph.D.  I worked at Imperva on machine learning problems related to cyber security.

Google Scholar / Linkedin / Twitter

Preprints

• Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers

Gilad Yehudai*, Clayton Sanford*, Maya Bechler-Speicher, Orr Fischer, Ran Gilad-Bachrach, Amir Globerson

Arxiv Preprint 

• Compositional Reasoning with Transformers, RNNs, and Chain of Thought

Gilad Yehudai, Noah Amsel, Joan Bruna

Arxiv Preprint 

• Reconstructing Training Data From Real World Models Trained with Transfer Learning

Yakir Oz, Gilad Yehudai, Gal Vardi, Itai Antebi, Michal Irani, Niv Haim

Arxiv Preprint 

• When Can Transformers Count to n?

Gilad Yehudai, Haim Kaplan, Asma Ghandeharioun, Mor Geva, Amir Globerson

Arxiv Preprint 

Publications

  * = Equal contribution

• Logarithmic Width Suffices for Robust Memorization

Amitsour Egosi, Gilad Yehudai, Ohad Shamir

COLT 2025

• On the Benefits of Rank in Attention Layers

Noah Amsel, Gilad Yehudai, Joan Bruna

ICLR 2025

• Locally Optimal Descent for Dynamic Stepsize Scheduling

Gilad Yehudai, Alon Cohen, Amit Daniely, Yoel Drori, Tomer Koren, Mariano Schain

AISTATS 2025

• MALT Powers Up Adversarial Attacks

Odelia Melamed, Gilad Yehudai, Adi Shamir

NeurIPS 2024

• On the Reconstruction of Training Data from Group Invariant Networks

Ran Elbaz, Gilad Yehudai, Meirav Galun, Haggai Maron

NeuReps Workshop, NeurIPS 2024

• RedEx: Beyond Fixed Representation Methods via Convex Optimization

Amit Daniely, Mariano Schain, Gilad Yehudai

ALT 2024‏

• Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspaces

Odelia Melamed, Gilad Yehudai, Gal Vardi‏

NeurIPS 2023 

• Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses‏

Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, Yakir Oz, Yaniv, Nikankin, Michal Irani

NeurIPS 2023

• From Tempered to Benign Overfitting in ReLU Neural Networks‏

Guy Kornowski*, Gilad Yehudai*, Ohad Shamir

NeurIPS 2023 (Spotlight)

• Reconstructing Training Data from Multiclass Neural Networks

Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, Michal Irani

ICLR 2023 Workshop for Trusted ML

• Reconstructing Training Data from Trained Neural Networks‏ 

Niv Haim*, Gal Vardi*, Gilad Yehudai*, Ohad Shamir, Michal Irani

Project page

NeurIPS 2022 (Oral presentation)

• Gradient Methods Provably Converge to Non-Robust Networks‏ 

Gal Vardi*, Gilad Yehudai*, Ohad Shamir

NeurIPS 2022

• Width is Less Important than Depth in ReLU Neural Networks

Gal Vardi*, Gilad Yehudai*, Ohad Shamir

COLT 2022

• On the Optimal Memorization Power of ReLU Neural Networks

Gal Vardi*, Gilad Yehudai*, Ohad Shamir

ICLR 2022 (spotlight)

• Learning a Single Neuron with Bias Using Gradient Descent‏

Gal Vardi*, Gilad Yehudai*, Ohad Shamir

NeurIPS 2021

• From Local Structures to Size Generalization in Graph Neural Networks

Gilad Yehudai, Ethan Fetaya, Eli Meiron, Gal Chechik, Haggai Maron

ICML 2021

• The Connection Between Approximation, Depth Separation and Learnability in Neural Networks

Eran Malach*, Gilad Yehudai*, Shai Shalev-Shwartz, Ohad Shamir

COLT 2021

• The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks

Itay Safran*, Gilad Yehudai*, Ohad Shamir 

COLT 2021

• Learning a Single Neuron with Gradient Methods

Gilad Yehudai, Ohad Shamir

COLT 2020

• Proving the Lottery Ticket Hypothesis: Pruning is All You Need

Eran Malach*, Gilad Yehudai*, Shai Shalev-Shwartz, Ohad Shamir

ICML 2020

• On the Power and Limitations of Random Features for Understanding Neural Networks

Gilad Yehudai, Ohad Shamir

NeurIPS 2019

Teaching

• Weizmann Institute of Science:

  • Topics in Machine Learning, Fall 2021 (TA)

  • Organized a student seminar about modern topics on deep learning, Fall 2021, 2022

• Tel-Aviv University:

  • Linear Algebra for engineers, Fall 2015, Fall 2016 (TA)

  • Calculus A for engineers, Spring 2016 (TA)

Awards

• 2024 - Menashe Milo Memorial Prize (from Weizmann) for excellent PhD thesis

• 2022 - Balvatnik Prize commendation 

• 2021 - VATAT fellowship for data science Ph.D. students 

• 2014 - Shefler scholarship given by TEVA

• 2014 - Excellence program in mathematics for Bs.c. students, given by Tel Aviv university

• 2013 - Memorial day scholarship given by Tel Aviv University

Patents

• Aggregating Alerts of Malicious Events for Computer Security  Publication: 2019

Gilad Yehudai, Itsik Mantin, Lior Fisch, Shelly Hershkovitz, Amichai Shulman, Moran Rachel Ambar

• Generating Security Rules Based on Collection Rules. Publication: 2020

Nadav Avital Arbel, Luda Lazar, Gilad Yehudai

Miscellaneous

• Some blog posts I wrote during my time at Imperva. 

• A podcast interview about the theory of deep learning (in Hebrew).

• I believe it's never too early to start studying machine learning.