My research interests span the areas of Control, Optimization, Algorithms and Applied Probability. I am interested in fundamental problems in Reinforcement Learning, Stochastic Optimization, Stochastic Networks, Queueing Theory, and Distributed Optimization. I use tools from these areas to work on problems such as scheduling and resource allocation and revenue optimization in a wide variety of systems including Data Centers, Cloud Computing, Wireless Networks, Block Chain as well as Ride hailing systems. Here is a list of selected publications.

For an exhaustive list of all my publications and patents in reverse chronological order, please see my CV. My Google Scholar Profile is here. Most of my recent papers can be found on arXiv here. Code from various projects can be found at this github page. 

Selected Publications

REINFORCEMENT Learning

Stochastic Approximation

• • Shaan Ul Haque, Sajad Khodadadian, Siva Theja Maguluri, "Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise," Arxiv version.

  • Zaiwei Chen, Siva Theja Maguluri, Martin Zubeldia, "Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise," Arxiv version. Video of a talk.

  • Zaiwei Chen, Shancong Mou, Siva Theja Maguluri, “Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterization,” ACM POMACS, conference version presented at ACM SIGMETRICS 2022. Arxiv Version. 

  • Zaiwei Chen, Siva Theja Maguluri, Sanjay Shakkottai, and Karthikeyan Shanmugam, "A Lyapunov Theory for Finite-Sample Guarantees of Markovian Stochastic Approximation," Operations Research (Arxiv version). 

    • Conference version:Zaiwei Chen, Siva Theja Maguluri, Sanjay Shakkottai, and Karthikeyan Shanmugam, “Finite-Sample Analysis of Stochastic Approximation Using Smooth Convex Envelopes,” in Neurips 2020 (Arxiv of Conf Version).

Value Function Methods

• • Zaiwei Chen, John-Paul Clarke, and Siva Theja Maguluri, "Target Network and Truncation Overcome the Deadly Triad in Q-Learning," SIMODS. Arxiv Version. 

  • Sheng Zhang, Zhe Zhang, Siva Theja Maguluri, "Finite Sample Analysis of Average-Reward TD Learning and Q-Learning," Neurips 2021.

  • Zaiwei Chen, Siva Theja Maguluri, Sanjay Shakkottai, and Karthikeyan Shanmugam, "Finite-Sample Analysis of Off-Policy TD-Learning via Generalized Bellman Operators," Neurips 2021. Arxiv version.

  • Zaiwei Chen, Sheng Zhang, Thinh Doan, John-Paul Clarke, Siva Theja Maguluri, “Finite-Sample Analysis of Nonlinear Stochastic Approximation with Applications in Reinforcement Learning,” Automatica. Arxiv vesion.

Policy Space Methods

• • Yashaswini Murthy, Isaac Grosof, Siva Theja Maguluri, R. Srikant "Performance of NPG in Countable State-Space Average-Cost RL," Arxiv version. 

  • Zaiwei Chen, Siva Theja Maguluri, "Sample Complexity of Policy-Based Methods under Off-Policy Sampling and Linear Function Approximation," AISTATS 2022. Arxiv version. Talk Link.

  • Zaiwei Chen, Sajad Khodadadian, Siva Theja Maguluri, "Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm with Linear Function Approximation," IEEE Control Systems Letters, Arxiv Version. 

  • Sajad Khodadadian, Zaiwei Chen, Siva Theja Maguluri, "Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm," ICML 2021, Arxiv version.

  • Sajad Khodadadian, Thinh Doan, Siva Theja Maguluri, Justin Romberg, "Finite Sample Analysis of Two-Time-Scale NaturalActor-Critic Algorithm," IEEE Transactions on Automatic Control. Arxiv version.

Multi Agent Reinforcement Learning

• Sajad Khodadadian, Pranay Sharma, Gauri Joshi, Siva Theja Maguluri, “Federated Reinforcement Learning: Communication-Efficient Algorithms and Convergence Analysis," ICML 2022. Arxiv version.
  Long presentation - 2.09% of all submissions.

• Thinh T. Doan, Siva Theja Maguluri, Justin Romberg, "Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation," SIAM Journal on Mathematics of Data Science. Conference version in ICML 2019. Arxiv version. 

Stochastic MATCHING NETWORKS

Matching (Two-Sided) Queues and Networks

• • Martin Zubeldia, Prakirt Raj Jhunjhunwala, Siva Theja Maguluri “Matching Queues with Abandonments in Quantum Switches: Stability and Throughput Analysis," Arxiv version. 

  • Sushil Mahavir Varma and Siva Theja Maguluri, “A Heavy Traffic Theory of Two-Sided Queues," Conference version in IFIP Performance 2021, Arxiv version.
    Stephen S. Lavenberg Best Student Paper Award at IFIP Performance 2021.
    Won the Alice and John Jarvis Ph.D. Student Paper Competition 2022 in ISyE.

  • Sushil Mahavir Varma and Siva Theja Maguluri, “Throughput Optimal Routing in Blockchain-Based Payment Systems,” IEEE Transactions on Control of Network Systems. Arxiv version.

Ride Hailing and Online Platforms

• Sushil Mahavir Varma, Francisco Castro, Siva Theja Maguluri, "Electric Vehicle Fleet and Charging Infrastructure Planning." Arxiv Version.
  Finalist in INFORMS TSL Student best paper competition 2023. 

• Sushil Mahavir Varma, Francisco Castro, Siva Theja Maguluri, “Near Optimal Control in Ride Hailing Platforms with Strategic Servers,” ACM SIGMETRICS 2021, Arxiv version.

• Sushil Mahavir Varma, Pornpawee Bumpensanti, Siva Theja Maguluri, He Wang, “Dynamic Pricing and Matching for Two-Sided Queues,” Operations Research. Conference version in ACM SIGMETRICS 2020. Arxiv version. 

STOCHASTIC PROCESSING Networks

Transform Method

• Prakirt Raj Jhunjhunwala, Daniela Hurtado Lange, Siva Theja Maguluri, "Exponential Tail Bounds on Queues: A Confluence of Non-Asymptotic Heavy Traffic and Large Deviations," Arxiv version. Video of a talk is here and slides are here. 

• Daniela Hurtado Lange, Siva Theja Maguluri, “Transform Methods for Heavy-Traffic Analysis,” Stochastic Systems 10 (4) 275-309. Arxiv version. Erratum. 

Data Center Networks and Switches

• • Prakirt Jhunjhunwala, Siva Theja Maguluri, "Heavy Traffic Distribution of Queueing Systems without Resource Pooling," Arxiv version. 

  • Shancong Mou, Siva Theja Maguluri, "Heavy Traffic Queue Length Behaviour in a Switch under Markovian Arrivals," Advanced in Applied Probability. Arxiv version.

  • Prakirt Raj Jhunjhunwala, Siva Theja Maguluri, "Low-Complexity Switch Scheduling Algorithms:Delay Optimality in Heavy Traffic,"  IEEE Transactions on Networking. Arxiv version.

  • Daniela Hurtado Lange, Siva Theja Maguluri, “Heavy-traffic Analysis of Queueing Systems with no Complete Resource Pooling," Mathematics of Operations Research. Conference version in ACM SIGMETRICS 2020, Arxiv version.
    Second place in INFORMS JFIG Best paper competition 2020.

  • Siva Theja Maguluri and R Srikant, “Heavy traffic queue length behavior in a switch under the MaxWeight algorithm,” Stochastic Systems, Vol. 6, (2016), 211-250 (electronic). ArXiv version here. Here are the slides that include this paper as well as the above one, and here is a video of one version of this talk.
    INFORMS APS Best paper award 2017.

Load Balancing

• • Prakirt Raj Jhunjhunwala, Martin Zubeldia, Siva Theja Maguluri, "Join-the-Shortest Queue with Abandonment: Critically Loaded and Heavily Overloaded Regimes," Arxiv version. 

  • Sushil Varma, Francisco Castro, Siva Theja Maguluri, "Power-of-d Choices Load Balancing in the Sub-Halfin Whitt Regime," Arxiv version.

  • Daniela Hurtado Lange, Siva Theja Maguluri, “A Load balancing system in the Many-Server Heavy-Traffic Asymptotics," Queueing Systems. A slightly different version based on a discrete-time model is on Arxiv.

  • Daniela Hurtado Lange, Sushil Mahavir Varma, Siva Theja Maguluri, "Logarithmic Heavy Traffic Error Bounds in Generalized Switch and Load Balancing Systems," Journal of Applied Probability. Arxiv version.

  • Daniela Hurtado Lange, Siva Theja Maguluri, “Throughput and Delay Optimality of Power-of-d Choices in Inhomogeneous Load Balancing Systems," Operations Research Letters. Arxiv version.

Other Stochastic Processing Networks

• • Sushil Mahavir Varma and Siva Theja Maguluri,  “Transportation Polytope and its Applications in Parallel Server Systems," Arxiv version. 

  • Weina Wang, Siva Theja Maguluri, R. Srikant, Lei Ying, “Heavy-Traffic Insensitive Bounds for Weighted Proportional Fair Bandwidth Sharing Policies,” Mathematics of Operations Research. Conference version in IFIP Performance 2017. Arxiv version.

  • Siva Theja Maguluri, R Srikant and Lei Ying, "Stochastic Models of Load Balancing and Scheduling in Cloud Computing Clusters," IEEE INFOCOM 2012. Technical Report is here, and the slides are here. 

Sponsors