Embedded Files
Agent Learning from Rich Feedback
Agent Learning from Rich Feedback
A personalization system learns about a user from their interactions. Counterfactual reasoning allowed us to learn from implicit but biased feedback like clicks. In the age of AI, feedback can be rich: users can tell an agent what they want in language, audio, or video. LLMs already aid personalization as priors, featurizers, and agents; this talk studies the reverse: personalization for LLM agents. Today's agents are powerful yet static "know-it-alls" that do not learn on the job. We developed techniques to train agents from rich feedback, which can be exponentially more sample-efficient than scalar rewards. Much of that feedback, however, is noisy, biased, and outside our control. We conjecture that causal reasoning will again pave the way forward: eliciting and valuing good feedback, and learning robustly from rich feedback.
Adith Swaminathan is a research scientist in the Machine Learning and Inference Research team at Netflix. He studies machine learning to improve decision-making for user-interactive systems, including recommender systems, LLM-augmented agents, interactive games and cloud computing systems. He received a PhD from Cornell University in 2017 and was a researcher at Microsoft Research Redmond from 2017-2024.
Toward Reliable Deployment of LLM Agents: From Uncertainty Quantification to Progress Advantage
Toward Reliable Deployment of LLM Agents: From Uncertainty Quantification to Progress Advantage
LLM agents are increasingly deployed on long-horizon tasks with tool use, irreversible actions, and unpredictable feedback. Yet we have few principled ways to tell, mid-episode, whether an agent is on track or quietly failing. Most uncertainty quantification (UQ) research still centers on single-turn QA, a poor match for interactive agents. In this talk, I'll present a general formulation of agent UQ and the challenges unique to agentic settings, from choosing uncertainty estimators to modeling how uncertainty evolves over an interaction. I'll then show that a powerful answer has been hiding in plain sight: RL post-training already yields an implicit step-level signal, the progress advantage, which recovers the optimal advantage function with no annotation or reward-model training. Across test-time scaling, UQ, and failure attribution, this free byproduct beats confidence baselines and even dedicated trained reward models.
Sharon Li is an Associate Professor in the Department of Computer Sciences at the University of Wisconsin-Madison. Her research focuses on algorithmic and theoretical foundations of reliable machine learning, addressing challenges in both model development and deployment in the open world. Previously she was a postdoc researcher in the Computer Science department at Stanford University. She completed her Ph.D. from Cornell University, advised by John E. Hopcroft. She serves as the Program Chair for ICML 2026. She was the recipient of Alfred P. Sloan Fellowship (2025), NSF CAREER Award (2023), MIT Innovators Under 35 Award (2023), AFOSR Young Investigator Award (2022), Forbes 30under30 in Science (2020), and multiple faculty research awards from Google, Meta, and Amazon. She was named the “Innovator of the Year 2023” by MIT Technology Review. Her works have won the Outstanding Paper Award at NeurIPS 2022 and ICLR 2022.
Page updated
Google Sites
Report abuse