Mark Rowland

I develop algorithms and analysis for reinforcement learning problems, from convergence theory through to large-scale applications of deep reinforcement learning.

Selected Recent Publications and Preprints

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
Mark Rowland, Li Kevin Wenliang, Rémi Munos, Clare Lyle, Yunhao Tang, Will Dabney
How many samples do you need to accurately estimate return distribution of a given policy? This paper shows that, roughly speaking, no more are required than for estimating just the value function of the policy.
arXiv preprint

An Analysis of Quantile Temporal-Difference Learning
Mark Rowland, Rémi Munos, Mohammad Gheshlaghi Azar, Yunhao Tang, Georg Ostrovski, Anna Harutyunyan, Karl Tuyls, Marc G. Bellemare, Will Dabney
Establishes convergence of quantile temporal-difference learning with probability 1, through stochastic approximation of differential inclusions and connections to quantile dynamic programming.
To appear in JMLR | arXiv preprint

The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation
Mark Rowland, Yunhao Tang, Clare Lyle, Rémi Munos, Marc G. Bellemare, Will Dabney
In tabular policy evaluation in stochastic environments, estimating value functions via quantiles can outperform the classical approach of TD learning due to a bias-variance trade-off.
ICML 2023 | arXiv version

Distributional Reinforcement Learning
Marc G. Bellemare, Will Dabney,  Mark Rowland
Textbook on distributional reinforcement learning.
Book website | MIT Press webpage (including open access copy)

See my Google Scholar page for full details of publications and preprints.

Contact
firstnamelastname [at] google [dot] com