Rana Shahout

rana@seas.harvard.edu

I work at the intersection of systems, machine learning, and algorithms, designing and building efficient, scalable AI systems. Recently, my work has focused on optimizing large language model (LLM) inference.

Short bio:

I am a Postdoctoral Fellow at Harvard University, working with Michael Mitzenmacher and Minlan Yu. I received my Ph.D. in Computer Science from the Technion, where I was advised by Roy Friedman. Before that, I worked at Mellanox (now NVIDIA) as a Senior Software Engineer. I also received my B.Sc. and M.Sc. from the Technion.

News:

Dec 25': I visited Boston University and gave an invited talk at the CISE Seminar on Prediction-Aware Algorithms for Efficient AI Systems.
Sept 25': MARS for fast inference for augmented LLMs has been accepted to NeurIPS 2025.
Sept 25': Two new papers are now available on arXiv: one on prediction-guided orchestration for LLM reasoning systems and another on load balancing in Mixture-of-Experts systems.
June 25': "Queueing, Predictions, and Large Language Models: Challenges and Open Problems" has been accepted to Stochastic Systems.
June 25': We presented a tutorial at SIGMETRICS 2025 on queueing systems, predictions, and LLM inference [slides].
April 25': Two papers, "Faster, Cheaper, Just as Good" and "Prefix and Output Length-Aware Scheduling for Efficient Online LLM Inference" will appear at the ICLR Workshop on Sparsity in LLMs (SLLM).
Mar 25': We uploaded a paper to arXiv on how queueing theory helps in understanding and improving LLM inference.
Jan 25': "DON’T STOP ME NOW:EMBEDDING BASED SCHEDULING FOR LLMS" has been accepted to ICLR 2025.
Oct 24': We uploaded a paper on fast inference for augmented LLMs to the arxiv.
Oct 24': "Distributed Recoverable Sketches" has been accepted to OPODIS 2024.
Oct 24': We uploaded a paper on embedding based scheduling for LLMs to the arxiv.
Oct 24': "Palimpzest: Optimizing AI-Powered Analytics with Declarative Query Processing" has been accepted to CIDR 2025.
Sept 24': "SkipPredict: When to Invest in Predictions for Scheduling" has been accepted to NeurIPS 2024.
Sept 24': "From Logs to Causal Inference: Diagnosing Large Systems" has been accepted to VLDB 2025.
Aug 24': "Learning-Augmented Frequency Estimation in Sliding Windows" has been accepted as a workshop to IEEE ICNP 2024.
Jul 24': "Learning-Based Heavy Hitters and Flow Frequency Estimation in Streams" has been accepted to IEEE ICNP 2024.
Jun 24': We uploaded a paper on learning-based heavy hitters and flow frequency estimation in streams to the arXiv.
Jun 24': "Press ECCS to Doubt (Your Causal Graph)" has been accepted to GUIDE-AI@SIGMOD 2024 and got the best paper award!
Mar 24': "Sawmill: From Logs to Causal Diagnosis of Large Systems" has been accepted as a demo to SIGMOD 2024.
Feb 24': We uploaded our paper on when to invest in predictions for scheduling to the arXiv.
Nov 23': "Sketching the Path to Efficiency: Lightweight Learned Cache Replacement" has been accepted to OPODIS 2023.
Jul 23': I have been awarded the Zuckerman Postdoctoral Fellowship
Jul 23': I have been awarded the Israeli Council for Higher Education Postdoctoral Fellowship
Jul 23': "Interface Design to Mitigate Inflation in Recommender Systems" has been accepted to RecSys 2023

Page updated

Google Sites

Report abuse

Rana Shahout

News:

Dec 25': I visited Boston University and gave an invited talk at the CISE Seminar on Prediction-Aware Algorithms for Efficient AI Systems.

Sept 25': MARS for fast inference for augmented LLMs has been accepted to NeurIPS 2025.

Sept 25': Two new papers are now available on arXiv: one on prediction-guided orchestration for LLM reasoning systems and another on load balancing in Mixture-of-Experts systems.

June 25': "Queueing, Predictions, and Large Language Models: Challenges and Open Problems" has been accepted to Stochastic Systems.

June 25': We presented a tutorial at SIGMETRICS 2025 on queueing systems, predictions, and LLM inference [slides].

April 25': Two papers, "Faster, Cheaper, Just as Good" and "Prefix and Output Length-Aware Scheduling for Efficient Online LLM Inference" will appear at the ICLR Workshop on Sparsity in LLMs (SLLM).

Mar 25': We uploaded a paper to arXiv on how queueing theory helps in understanding and improving LLM inference.

Jan 25': "DON’T STOP ME NOW:EMBEDDING BASED SCHEDULING FOR LLMS" has been accepted to ICLR 2025.

Oct 24': We uploaded a paper on fast inference for augmented LLMs to the arxiv.

Oct 24': "Distributed Recoverable Sketches" has been accepted to OPODIS 2024.

Oct 24': We uploaded a paper on embedding based scheduling for LLMs to the arxiv.

Oct 24': "Palimpzest: Optimizing AI-Powered Analytics with Declarative Query Processing" has been accepted to CIDR 2025.

Sept 24': "SkipPredict: When to Invest in Predictions for Scheduling" has been accepted to NeurIPS 2024.

Sept 24': "From Logs to Causal Inference: Diagnosing Large Systems" has been accepted to VLDB 2025.

Aug 24': "Learning-Augmented Frequency Estimation in Sliding Windows" has been accepted as a workshop to IEEE ICNP 2024.

Jul 24': "Learning-Based Heavy Hitters and Flow Frequency Estimation in Streams" has been accepted to IEEE ICNP 2024.

Jun 24': We uploaded a paper on learning-based heavy hitters and flow frequency estimation in streams to the arXiv.

Jun 24': "Press ECCS to Doubt (Your Causal Graph)" has been accepted to GUIDE-AI@SIGMOD 2024 and got the best paper award!

Mar 24': "Sawmill: From Logs to Causal Diagnosis of Large Systems" has been accepted as a demo to SIGMOD 2024.

Feb 24': We uploaded our paper on when to invest in predictions for scheduling to the arXiv.

Nov 23': "Sketching the Path to Efficiency: Lightweight Learned Cache Replacement" has been accepted to OPODIS 2023.

Jul 23': I have been awarded the Zuckerman Postdoctoral Fellowship

Jul 23': I have been awarded the Israeli Council for Higher Education Postdoctoral Fellowship

Jul 23': "Interface Design to Mitigate Inflation in Recommender Systems" has been accepted to RecSys 2023