Learning Exploration Strategies to Solve Real-World Marble Runs
Learning Exploration Strategies to Solve Real-World Marble Runs
Embedded Files
Tasks involving complex dynamic interactions remain challenging in robotics, because small variations in the environment can have a significant impact on task outcomes. For such tasks, learning a single deterministic policy that performs well across a variety of tasks and uncertain dynamics is difficult, so we focus on structuring exploration with multiple stochastic policies based on a mixture of experts (MoE) policy representation that can be efficiently adapted in the real world. The MoE policy is composed of stochastic sub-policies that allow exploration of multiple distinct regions of the action space (or strategies) and a high-level selection policy to guide exploration towards the most promising regions.
We develop a robot system to evaluate our approach in a real-world physical problem solving domain. After training the MoE policy in simulation, online learning in the real world demonstrates efficient adaptation within just a few dozen attempts. Our results confirm that representing multiple strategies promotes efficient adaptation in new environments and strategies learned under different dynamics can still provide useful information about where to look for solutions.
Page updated
Google Sites
Report abuse