Following the official recommendations, RSS and its workshops will take place virtually. Our workshop will be held at the same day and time (July 13, morning) in a virtual manner, keeping the discussions and presentations live. Contributed papers will be presented as highlight virtual presentations during the workshop.
When humans learn new tasks, they connect sensory inputs to actions to create sensory-motor loops. These loops are not always best learned in the same action space. While some tasks are easy to learn and control in joint space, others are easier to define in Cartesian or other task spaces. Even more, some tasks require not only to learn motion but also other dynamic components like the amount of force to apply on the environment.
In robot learning, for example in reinforcement learning, the goal is also to learn sensorimotor loops with a policy that maps observations to actions, elements of some action space. Given that many robotic platforms take desired joint torques as robot commands, a possibility is to learn directly a map ping from sensor signals to torques (i.e. “from pixels to torques”). However, this setup adds complexity to the learning problem since the task can be more easily learned in a different action space.
Recently, several efforts explore how to compose analytical controllers into robot learning frameworks as a way to factorize the learning problem into policy learning and control. The policy still learns to map observations to actions that are reference signals to the controller; a task where methods like Deep RL have succeeded even using high dimensional inputs images.
These actions are mapped into low level robot commands by the analytic controller. The controller, thus, enables a different action space for robot learning. These recent studies confirmed the intuition that different action spaces facilitate learning, depending on the characteristics of the task to learn.
In this workshop we propose to study further this new paradigm that bridges robot learning and advanced robot control. We will try to shed light on questions such as: what properties of manipulation tasks indicate the best action space for robot learning? How to best incorporate compliant and force control into learning algorithms? Do the most common action spaces emerge as natural representation to learn tasks?
Call for contributions
We invite all authors with work relevant to the topic to submit a four-page abstract (using RSS format) June 8, 2020 through the CMT system here. The abstracts will be reviewed in a single-blind process.
Authors will be notified of acceptance by June 15, 2020. Selected contributions will be highlighted as short spotlight presentations during the workshop.
All times are PST:
09:50 AM - 10:50 AM Paper spotlight presentations + Q&A (video)
11:50 AM - 12:30 PM Panel discussion (video)
Leading theories of motor control assume that the motor system learns probabilistic models and that motor behavior can be explained as the optimization of payoff or cost criteria under the expectation of these models. Here we discuss how the motor system exploits task variability to build up efficient abstractions through structural learning and how the learning of such abstractions may be guided both by external costs and internal information processing costs. We compare human behavior to such optimal models and discuss in how far sensorimotor abstractions may subserve meta-learning behavior.
What should be learned, and what assumed known about our world, is among the core questions of robotics and AI in general. I will first discuss work on physical manipulation planning that fully builds on carefully formulated analytical models of our world. Once we have such purely model-based solvers, they open great opportunities for learning. I will briefly discuss learning to reason much more efficiently, as well as learning to execute plans.
Meta-learning is a promising route towards enabling fast learning of new skills by transferring knowledge from previously learned behaviors. In this talk I will review some of the existing meta-learning approaches with a focus on which actions-space they utilize. I will then highlight the opportunities of learning loss functions for fast adaptation of dynamics models.
Legged locomotion is a rich continuous control task having many possible action spaces that could be used for control: torques, muscle activations, PD-targets, ground-reaction forces, and learned action abstractions. Which give the best performance? Which are best for learning? Which transfer best to hardware? We describe our past and ongoing work towards these problems, including direct comparisons of learning in various action spaces, and learned solutions to difficult locomotion tasks. Finally, we argue that action spaces are fully intertwined with the larger problem of task abstraction.
Karlsruhe Institute of Technology
University of Toronto
NYU and MPI