Optional Reading

PyTorch:

• Deep Learning with PyTorch: A 60 Minute Blitz

• PyTorch Tutorials

• Example PyTorch code: https://github.com/yunjey/pytorch-tutorial

Reinforcement learning taxonomy: Reinforcement learning improves behaviour from evaluative feedback

Multi-armed bandits:

• Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto, Chapter 2

• Bandit Algorithms by Tor Lattimore and Csaba Szepesv´ari, Chapters 1, 4-7, 11, 19 (Introduction to bandits; regret; concentration inequalities; Explore-then-Commit; Upper confidence bounds, EXP3, linUCB)

• Thompson Sampling

  • Blog post

  • Table of Conjugate Distributions

  • A Tutorial on Thompson Sampling

• linUCB

  • A Contextual-Bandit Approach to Personalized News Article Recommendation

Model-free Reinforcement learning: Q-function based methods

• Reinforcement Learning: An Introduction by Richard Sutton, Ch. 3-7, 12

Model-free Reinforcement learning: Policy Gradients

• Reinforcement Learning: An Introduction by Richard Sutton, Ch. 13

• Introduction

• Kinds of RL algorithms

• Policy gradients

• PPO

• Generalized Advantage Estimation (GAE)

Model-based Reinforcement Learning

• PDDM: Deep Dynamics Models for Learning Dexterous Manipulation

• PETS: Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models

• TOPDM: Solving Challenging Dexterous Manipulation Tasks With Trajectory Optimisation and Reinforcement Learning

Learning from demonstration

• DAgger

• Inverse reinforcement learning (IRL):

  • Linear programming IRLR: LP-IRL

  • Maximum entropy IRL: MaxEntIRL

Offline RL:

• Advantage Weighted Actor Critic: AWAC

• Offline RL with Implicit Q-learning (IQL)

• Conservative Q-learning (CQL)

• Latent Action Space for Offline RL: PLAS