# Mark Rowland

Research Scientist | DeepMind

## About Me

I'm a Research Scientist at DeepMind. I primarily work on reinforcement learning, particularly on problems relating to distributional reinforcement learning and off-policy learning.

I'm also interested in many areas across statistics, probability, optimisation, and game theory, as well as interactions of these fields with areas of pure maths such as group theory.

Prior to joining DeepMind, I was a PhD student at Clare College, Cambridge University, co-supervised by Rich Turner at the Machine Learning Group and John Aston at Statslab. Before my PhD, I studied for a BA and MMath at Cambridge.

## News

**(Jan 2021)** *On the Effect of Auxiliary Tasks on Representation Dynamics* will appear at AISTATS 2021.

**(Dec 2020)** *The Value-Improvement Path* will appear at AAAI 2021.

## Publications and Preprints

**Revisiting Peng’s Q(λ) for Modern Reinforcement Learning**

Tadashi Kozuno*, Yunhao Tang*, Mark Rowland, Remi Munos, Steven Kapturowski, Will Dabney, Michal Valko, David Abel [*equal contribution]

**On the Effect of Auxiliary Tasks on Representation Dynamics**

Clare Lyle*, Mark Rowland*, Georg Ostrovski, Will Dabney [*equal contribution]

To appear at *AISTATS 2021*

**The Value-Improvement Path: Towards Better Representations for Reinforcement Learning**

Will Dabney, André Barreto, Mark Rowland, Robert Dadashi, John Quan, Marc G. Bellemare, David Silver

*AAAI 2021*

**Revisiting Fundamentals of Experience Replay**

William Fedus*, Prajit Ramachandran*, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, Will Dabney [*equal contribution]

*ICML 2020*

**Fast Computation of Nash Equilibria in Imperfect Information Games**

Rémi Munos, Julien Pérolat, Jean-Baptiste Lespiau, Mark Rowland, Bart De Vylder, Marc Lanctot, Finbarr Timbers, Daniel Hennes, Shayegan Omidshafiei, Audrunas Gruslys , Mohammad Gheshlaghi Azar, Edward Lockhart, Karl Tuyls

*ICML 2020*

**Navigating the Landscape of Games**

Shayegan Omidshafiei*, Karl Tuyls*, Wojciech M. Czarnecki, Francisco C. Santos, Mark Rowland, Jerome Connor, Daniel Hennes, Paul Muller, Julien Pérolat, Bart De Vylder, Audrunas Gruslys, Rémi Munos [*equal contribution]

*Nature Communications, 2020*

**From Poincaré Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization**

Julien Pérolat, Rémi Munos, Jean-Baptiste Lespiau, Shayegan Omidshafiei, Mark Rowland, Pedro Ortega, Neil Burch, Thomas Anthony, David Balduzzi, Bart De Vylder, Georgios Piliouras, Marc Lanctot, Karl Tuyls

**Adaptive Trade-Offs in Off-Policy Learning**

Mark Rowland*, Will Dabney*, Rémi Munos [*equal contribution]

*AISTATS 2020*

**Conditional Importance Sampling for Off-Policy Learning**

Mark Rowland, Anna Harutyunyan, Hado van Hasselt, Diana Borsa, Tom Schaul, Rémi Munos, Will Dabney

*AISTATS 2020*

**A Generalized Training Approach for Multiagent Learning**

Paul Muller, Shayegan Omidshafiei, Mark Rowland, Karl Tuyls, Julien Pérolat, Siqi Liu, Daniel Hennes, Luke Marris, Marc Lanctot, Edward Hughes, Zhe Wang, Guy Lever, Nicolas Heess, Thore Graepel, Rémi Munos

*ICLR 2020*

**Multiagent Evaluation under Incomplete Information**

Mark Rowland*, Shayegan Omidshafiei*, Karl Tuyls, Julien Pérolat, Michal Valko, Georgios Piliouras, Rémi Munos [*equal contribution]

*NeurIPS 2019*

**α-Rank: Multi-Agent Evaluation by Evolution**

Shayegan Omidshafiei*, Christos Papadimitriou*, Georgios Piliouras*, Karl Tuyls*, Mark Rowland, Jean-Baptiste Lespiau, Wojciech M. Czarnecki, Marc Lanctot, Julien Pérolat, Rémi Munos [*equal contribution]

*Scientific Reports, 2019*

**Meta-learning of Sequential Strategies**

Pedro A. Ortega, Jane X. Wang, Mark Rowland, Tim Genewein, Zeb Kurth-Nelson, Razvan Pascanu, Nicolas Heess, Joel Veness, Alex Pritzel, Pablo Sprechmann, Siddhant M. Jayakumar, Tom McGrath, Kevin Miller, Mohammad Azar, Ian Osband, Neil Rabinowitz, András György, Silvia Chiappa, Simon Osindero, Yee Whye Teh, Hado van Hasselt, Nando de Freitas, Matthew Botvinick, Shane Legg

*DeepMind Tech Report, 2019*

**Statistics and Samples in Distributional Reinforcement Learning**

Mark Rowland, Robert Dadashi, Saurabh Kumar, Rémi Munos, Marc G. Bellemare, Will Dabney

*ICML 2019*

**Unifying Orthogonal Monte Carlo Methods**

Krzysztof Choromanski*, Mark Rowland*, Wenyu Chen, Adrian Weller [*equal contribution]

*ICML 2019*

**Antithetic and Monte Carlo Kernel Estimators for Partial Rankings**

Maria Lomeli, Mark Rowland, Arthur Gretton, Zoubin Ghahramani

*Statistics and Computing, 2019*

**Orthogonal Estimation of Wasserstein Distances**

Mark Rowland*, Jiri Hron*, Yunhao Tang*, Krzysztof Choromanski, Tamas Sarlos and Adrian Weller [*equal contribution]

*AISTATS 2019*

**Gaussian Process Behaviour in Wide Deep Neural Networks**

Alexander G. de G. Matthews, Mark Rowland, Jiri Hron, Richard E. Turner, Zoubin Ghahramani

*Extended version of the paper appearing at ICLR 2018*

**Geometrically Coupled Monte Carlo Sampling**

Mark Rowland*, Krzysztof Choromanski*, François Chalus, Aldo Pacchiano, Tamas Sarlos, Richard E. Turner, Adrian Weller [*equal contribution]

*NeurIPS 2018*

**Structured Evolution with Compact Architectures for Scalable Policy Optimization**

Krzysztof Choromanski*, Mark Rowland*, Vikas Sindhwani, Richard E. Turner, Adrian Weller [*equal contribution]

*ICML 2018*

**Gaussian Process Behaviour in Wide Deep Neural Networks**

Alexander G. de G. Matthews, Jiri Hron, Mark Rowland, Richard E. Turner, Zoubin Ghahramani

*ICLR 2018*

[paper] [see above for extended version]

**An Analysis of Categorical Distributional Reinforcement Learning**

Mark Rowland, Marc G. Bellemare, Will Dabney, Rémi Munos, Yee Whye Teh

*AISTATS 2018*

**The Geometry of Random Features**

Krzysztof Choromanski*, Mark Rowland*, Tamas Sarlos, Vikas Sindhwani, Richard E. Turner, Adrian Weller [*equal contribution]

*AISTATS 2018*

**Distributional Reinforcement Learning with Quantile Regression**

Will Dabney, Mark Rowland, Marc G. Bellemare, Rémi Munos

*AAAI 2018*

**Uprooting and Rerooting Higher-Order Graphical Models**

Mark Rowland*, Adrian Weller* [*equal contribution]

*NIPS 2017*

**The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings**

Krzysztof Choromanski*, Mark Rowland*, Adrian Weller [*equal contribution]

*NIPS 2017*

**Magnetic Hamiltonian Monte Carlo**

Nilesh Tripuraneni, Mark Rowland, Zoubin Ghahramani, Richard E. Turner

*ICML 2017*

[paper] [supplementary material] [poster]

**Conditions Beyond Treewidth for Tightness of Higher-Order LP Relaxations**

Mark Rowland, Aldo Pacchiano, Adrian Weller

*AISTATS 2017*

[paper] [supplementary material] [poster]

Also presented at the International Conference on Principles and Practices of Constraint Programming (CP 2017)

**Black-box Alpha Divergence Minimization**

Yingzhen Li, José Miguel Hernández-Lobato, Mark Rowland, Daniel Hernández-Lobato, Thang Bui, Richard E. Turner

*ICML 2016*

[paper] [supplementary material]

**Tightness of LP Relaxations for Almost Balanced Models**

Adrian Weller, Mark Rowland, David Sontag

*AISTATS 2016*

[paper] [supplementary material] [poster]

Also presented at the International Conference on Principles and Practices of Constraint Programming (CP 2016)

## Teaching

During my PhD, I gave supervisions on Part IA Groups, Part IA Probability, Part IB Linear Algebra, and Part IB Statistics. I also gave examples classes for Part III Bayesian Modelling and Computation in Lent 2017, and ran MATLAB introduction sessions for Maths Tripos students.

## Contact

firstnamelastname [at] google.com