Abstract: We study the problem of average reward Multi-Agent Reinforcement Learning (MARL) where agents interact within a network. Each agent in the network conducts local updates with information within its $k$-hop neighborhood and collaboratively maximizes the overall reward of the entire network. We first provide impossibility results in achieving convergence to the global optimal joint policies in our setting. These impossibility results highlight the challenges of decentralized learning with local information in multi-agent systems, namely the issues of \emph{multiagency}, where each agent independently optimizes its policy, and \emph{partial observability}, where agents have limited visibility into the global state. Given these challenges which indicate that decentralized policy optimization can generally be certified only up to stationarity, we provide finite-sample convergence guarantees for a decentralized actor-critic algorithm with linear function approximation that converges to a locally optimal solution with small gradient norms. To overcome such impossibility results, we further show that adding fixed-state entropy regularization reshapes the objective landscape which leads to even stronger global convergence guarantees for our proposed actor-critic algorithm.

Biography: Yashaswini Murthy is a postdoctoral scholar in Computing and Mathematical Sciences at Caltech. She will join the University of Texas at Austin as an Assistant Professor in Operations Research and Industrial Engineering in Fall 2026. She received her Ph.D. in Electrical and Computer Engineering from the University of Illinois Urbana-Champaign. Prior to that, she earned her Bachelor’s and Master’s degrees in Mechanical Engineering from the Indian Institute of Technology Bombay. Her research focuses on long-term decision-making, with an emphasis on average-reward formulations and distributional robustness. Her work has been recognized with several honors, including the Mavis Future Faculty Fellowship and the Joan and Lalit Bahl Fellowship from UIUC, as well as recognition as a Rising Star in EECS and in ISyE–MS&E–IOE.

Abstract: Reward inference (learning a reward model from human preferences) is a critical intermediate step in Preference-based Reinforcement Learning (PbRL), such as Reinforcement Learning from Human Feedback (RLHF) for fine-tuning Large Language Models (LLMs). In practice, reward inference faces fundamental challenges such as distribution shift, reward model overfitting, and problem misspecification. An alternative approach is direct policy optimization without reward inference, such as Direct Preference Optimization (DPO), which offers a much simpler pipeline but only works in the bandit setting or for deterministic MDPs. This talk introduces new algorithms for stochastic MDPs and general preference models (link functions). The key idea is a sign-based policy perturbation approach based on zeroth-order optimization. We will discuss its applications to unknown link functions and federated RLHF. 

Bio: Lei Ying is a Professor in the Electrical Engineering and Computer Science Department at the University of Michigan, Ann Arbor. He is an IEEE Fellow and an Editor-at-Large for the IEEE/ACM Transactions on Networking. His research focuses on the interplay between complex stochastic systems and big data, including reinforcement learning, large-scale communication/computing systems for big-data processing, private data marketplaces, and large-scale graph mining.

Abstract: We derive instance-dependent tail bounds for the regret of optimism-based reinforcement learning in finite-horizon tabular Markov decision processes with unknown transition dynamics. We first study a UCBVI-type (model-based) algorithm and characterize the tail distribution of the cumulative regret over all episodes via explicit bounds on the probability of the regret exceeding a specific value, going beyond analyses limited to expected regret or a single high-probability quantile. We analyze two natural exploration-bonus schedules for UCBVI: (i) a schedule dependent on the total number of episodes that explicitly incorporates this total, and (ii) an anytime schedule (independent of the total) that depends only on the current episode index. We then complement the model-based results with an analysis of optimistic Q-learning (model-free) under a schedule dependent on the total number of episodes. 

Across both the model-based and model-free settings, we obtain upper bounds on the probability of the regret exceeding a given value, revealing a distinctive two-regime structure: a sub-Gaussian tail starting from an instance-dependent scale up to a transition threshold, followed by a sub-Weibull tail beyond that point. We further derive corresponding instance-dependent bounds on the expected cumulative regret. The proposed algorithms depend on a tuning parameter, alpha, which balances the expected regret and the range over which the regret exhibits sub-Gaussian decay.

Bio: Mehrdad Moharrami is an Assistant Professor in Computer Science at the University of Iowa. Previously, he was a TRIPODS Postdoctoral Research Fellow at the University of Illinois at Urbana Champaign. He received his B.Sc. from the Sharif University of Technology, Iran, in Mathematics, as well as Electrical Engineering. He holds an M.Sc. in Electrical Engineering and an M.Sc. in Mathematics from the University of Michigan. He received his Ph.D. in Electrical Engineering in Winter 2020, for which he was awarded the Rackham Predoctoral Fellowship for an outstanding dissertation. His research interests are Markov decision processes, reinforcement learning, and random graph models for economics, learning, and computation. 

Abstract: Decision-making artificial intelligence (AI) has revolutionized human life ranging from healthcare, daily life, to scientific discovery. However, current AI systems often lack reliability and are highly vulnerable to small changes in complex, interactive, and dynamic environments. My research focuses on achieving both reliability and learning efficiency simultaneously when building AI solutions. These two goals seem conflicting, as enhancing robustness against variability often leads to more complex problems that requires more data and computational resources, at the cost of learning efficiency. But does it have to?

In this talk, I overview my work on building reliable decision-making AI without sacrificing learning efficiency, offering insights into effective optimization problem design for reliable AI. To begin, I will focus on reinforcement learning (RL) — a key framework for sequential decision-making, and demonstrate how distributional robustness can be achieved provably without paying statistical premium (additional training data cost) compared to non-robust counterparts. Next, shifting to decision-making in strategic multi-agent systems, I will demonstrate that incorporating realistic risk preferences—a key feature of human decision-making—enables computational tractability, a benefit not present in traditional models. Finally, I will present some interesting future directions for building reliable, learning-efficient AI solutions for human-centered applications, though agentic and multi-agentic AI systems.

Bio: Laixi Shi is an Assistant Professor of Electrical and Computer Engineering at Johns Hopkins University and is affiliated with the Data Science and Artificial Intelligence Institute. She received her Ph.D. in Electrical and Computer Engineering from Carnegie Mellon University (CMU) in August 2023; her dissertation received the CMU ECE A.G. Milnes Award. Prior to joining Johns Hopkins, she was a postdoctoral fellow in the Department of Computing and Mathematical Sciences at the California Institute of Technology (Caltech). She holds a B.S. in Electronic Engineering from Tsinghua University (2014–2018) and has conducted research with the Google Research Brain Team and Mitsubishi Electric Research Laboratories. Her research focuses on robust and data-efficient decision making at the intersection of data science, optimization, and statistics, with an emphasis on reinforcement learning and inverse problems. Her work spans both theoretical foundations and practical applications. She has been honored with five Rising Star awards across electrical engineering, computer science, machine learning, signal processing, and computational data science. She serves as a Program Committee Member for Learning for Dynamics & Control Conference (L4DC) and area chair for Conference on Parsimony and Learning (CPAL).

Abstract: We study a multi-class single-server (M/M/1) queue with non-idling scheduling control and long-run average holding costs, and develop a formulation that connects queueing control to stochastic optimal transport (SOT) through the Basic Adjoint Relationship (BAR). We call this the BAR-SOT. Under a stationary control policy u, the controlled queue-length process is a continuous-time Markov chain whose invariant distribution (πᵘ) satisfies the BAR, i.e., a weak stationarity condition (∑ₓ πᵘ(x) Gᵘ f(x) = 0) for all test functions f. We show that this constraint is the discrete-event, stationary version of Toshio Mikami’s weak Fokker–Planck constraints in SOT, allowing the scheduling problem to be recast as a BAR-constrained SOT in which the Markov transition kernel acts as a self-coupling transport plan for the stationary measure (πᵘ). The resulting optimization is linear in πᵘ with an infinite family of weak constraints, and admits a dual characterization in terms of the average-cost Poisson's equation. This perspective yields a geometric interpretation of optimal scheduling as a gradient-flow-like selection rule driven by discrete potentials, recovers the classical cμ-rule under linear costs, and predicts state-dependent switching surfaces under nonlinear costs. This is joint work with Sharan Srinivasan and Berke Turkay. 

Bio: Harsha Honnappa is an Associate Professor in the Edwardson School of Industrial Engineering at Purdue University, where he directs the Stochastic Systems Lab. He is an applied probabilist with interests in the modeling and analysis of stochastic systems, theoretical statistics, stochastic optimization and control. His research is supported by multiple grants from the National Science Foundation, including an NSF CAREER award, the Office of Naval Research, the Purdue Research Foundation, and through the Edwardson School of Industrial Engineering’s Frontiers awards. He is currently Area Editor for Stochastic Models for Operations Research Letters, and Associate Editor at Operations Research and Queueing Systems Journals.