Abstract: Reinforcement learning (RL) has become a central approach to post-training large language models, driving advances in chain-of-thought (CoT) reasoning and agentic problem solving. A striking feature of this approach is that rewards based only on final outcomes can improve reasoning across many intermediate steps, without explicitly supervising those steps. This raises fundamental challenges in understanding how sparse feedback guides learning, produces complex behaviors, and supports generalization. Despite its practical importance, a theoretical understanding of this process remains limited. 

In this talk, I will present our recent work toward a theory of reinforcement learning for reasoning in transformers, revealing how training dynamics give rise to multi-step reasoning and search capabilities. We identify curriculum structure as a key factor in progressing from simple skills to complex behaviors, including an implicit curriculum that emerges from training on mixed-difficulty problems. We characterize how the difficulty spectrum governs steady progress versus grokking-like transitions, with prolonged plateaus followed by abrupt improvements. We further explain how RL shapes specialized attention mechanisms that equip transformer-based policies with exploration, memory, and backtracking, yielding depth-first search strategies that generalize from shallow training trees to deeper ones. These findings connect curriculum structure, optimization dynamics, and attention mechanisms, providing new insights into how RL develops reasoning and agentic capabilities from sparse feedback.

Bio: Dr. Yingbin Liang is currently a Professor at the Department of Electrical and Computer Engineering at the Ohio State University (OSU), and a core faculty of the Ohio State Translational Data Analytics Institute (TDAI). She also serves as the Deputy Director of the NSF AI-EDGE Institute and the Co-Lead for Foundational AI Pillar of OSU AI (X) Hub. Dr. Liang received the Ph.D. degree in Electrical Engineering from the University of Illinois at Urbana-Champaign in 2005, and served on the faculty of University of Hawaii and Syracuse University before she joined OSU. Dr. Liang's research lies at the intersection of machine learning, large-scale optimization, statistical signal processing, information theory, and wireless networks, with their growing applications to other scientific domains. She received the National Science Foundation CAREER Award and the State of Hawaii Governor Innovation Award in 2009. She also received EURASIP Best Paper Award in 2014. She is currently an Information Theory Society Distinguished Lecturer for 2026–2027. Dr. Liang is an IEEE fellow.