Joshua Zhanson*, Emilio Parisotto*, Ruslan Salakhutdinov
{jzhanson, eparisot, rsalakhu}@cs.cmu.edu
*equal contribution
Workshop on Structure & Priors in Reinforcement Learning @ International Conference on Learning Representations 2019
In this work, we explore incorporating spatial information into feature and action representations of deep reinforcement learning in the pursuit of a single multi-body locomotion policy. Instead of the traditional feature vectors used in continuous RL, we introduce a body-space state representation that maps sensor readings onto a spatial grid overlay of the robot's body, explicitly encoding relative positional information. Additionally, we introduce a motor-space action representation that projects motor torques out of a spatial grid. Models map from input body-space to output motor-space, instead of from the observation space of joint velocities and angles to the action space of joint torques. To demonstrate the multitask and transfer capabilities of models trained with our representations, we introduce an environment based on the Box2D physics simulator that allows creation of robot bodies with arbitrary structure, physical properties, and dimensions and show that models trained using our representations can learn a walking policy transferable across many randomized body configurations.
