Abstract:  I will discuss machine learning surrogates for discrete time dynamic programming, motivated by optimal order execution with dynamic parametric uncertainty. Our numerical algorithm utilizes an actor-critic framework to construct two neural-network (NN) surrogates for the value function and the feedback control. Our base model features discrete time, stochastic transient price impact generalizing Obizhaeva and Wang (2013). We first consider learning the optimal strategy across a range of model configurations, including price impact and resilience parameters, as well as initial stochastic states, giving a state space in anywhere between 2 and 6 dimensions. We then apply the lens of adaptive robust stochastic control to consider online statistical learning of model parameters along with a worst-case min-max optimization. Thus, the controller is dynamically learning model parameters based on her observations while explicitly accounting for Bayesian uncertainty of the parameter estimates via a posterior uncertainty set, yielding a 7-dimensional augmented state space. We show that our approach straightforwardly extends to cover such state-dependent min-max optimization, and similarly can be adjusted for alternative control frameworks, such as adaptive or static robust formulations. Our work includes publicly available Jupyter notebooks and provides a pedagogical testbed for actor-critic frameworks for stochastic control.  

This is joint work with Moritz Voss (UCLA) and Tao Chen (QRM).

Bio: Mike Ludkovski is a Professor of Statistics and Applied Probability at the University of California Santa Barbara where he co-directs the Center for Financial Mathematics and Actuarial Research. Among his research interests are Monte Carlo techniques for optimal stopping/stochastic control, modeling of renewable energy markets, Gaussian process models for quantitative finance, and mortality analysis. His research has been supported by NSF, DOE, ARPA-E and CAS.  He holds a Ph.D. in Operations Research and Financial Engineering from Princeton University and has held visiting positions at the London School of Economics and Paris Dauphine University.

Meeting Recording: https://ucsb.zoom.us/rec/share/EpxwV1DI7-yqTTAnMOH51ijMWPwWdbRp3b4BKFSigmN4MSkHB0rEMVxUldnye96S.JxgC9AXvKSAV-lEh

Access Passcode: kP0kji=K