Abstract: Reinforcement learning (RL) is a largely model-free approach that learns directly the best policies (instead of learning a model) based on only samples (i.e. data) - either exogenously given or strategically generated. It mimics human's - especially children's - learning process through trial-and-error interaction with the unknown environment. RL has a vast literature but mainly limited in Markov Decision Processes (MDPs) in discrete time and space, and many RL algorithms lack theoretical foundations and hence lack interpretability. RL in continuous time and space is both practically motivated and theoretically promising. The related research is still in its infancy, but the results so far not only lay rigorous theoretical foundations for some well-known MDP RL algorithms but, more importantly, provide an entirely new perspective to look at the RL problems. This talk gives a review of some of these recent results, including how to optimally generate the trial-and-error strategies, how to evaluate a given policy based on only samples, how to improve the current policy, and what is the proper Q-function in the continuous setting.

Bio: Xunyu Zhou is the Liu Family Professor of Financial Engineering and the Director of the Nie Center for Intelligent Asset Management at Columbia University. He was the Nomura Professor of Mathematical Finance at the University of Oxford before joining Columbia in 2016.

His research covers stochastic control, dynamic portfolio selection, asset pricing, behavioral finance, and time inconsistency. Currently, his research focuses on continuous-time reinforcement learning and applications to optimization broadly and to wealth management specifically. He is a recipient of the Wolfson Research Award from The Royal Society, the Outstanding Paper Prize from SIAM, the Alexander von Humboldt Research Fellowship, and the Croucher Senior Research Fellowship. He was an invited speaker at the 2010 International Congress of Mathematicians, a Humboldt Distinguished Lecturer at Humboldt University and an Archimedes Lecturer at Columbia. He is both an IEEE Fellow and a SIAM Fellow.

Xunyu Zhou received his PhD in Operations Research and Control Theory from Fudan University in 1989.

Meeting Recording: https://ucsb.zoom.us/rec/share/ehNlF6-jSNJ6KvuYttsOutINFI_XgkO5XSvoQbIPvNfN61JDQIiBDQywsUCEnQaP.1hyEy0tT-evBfdBq

Access Passcode: !6B8HsX.