Learning to Adapt:

Meta-Learning for Model-Based Control

Ignasi Clavera*, Anusha Nagabandi*, Pieter Abbeel, Sergey Levine, Chelsea Finn

(View Paper Here )

Online adaptation is crucial for systems deployed into the real world; this skill requires the ability to use prior knowledge to quickly adapt to new tasks and environmental perturbations. In this work, we propose a meta-learning approach for learning to adapt online. In order to achieve low sample complexity in the meta-training phase, as is required for real-world applications, we study the online adaptation problem in the context of model-based reinforcement learning. Our approach efficiently meta-learns a global dynamics model, which can be combined with recent experience for fast, online adaptation. We demonstrate successful online adaptation on several simulated robotic control tasks with complex contact dynamics.

1. ANT (Trained on random legs being crippled)

1a. Tested on a crippled leg that was seen as crippled during training

*ours*

model-based baseline

(no adaptation)

model-free oracle (trained only on this particular scenario + with lot more training data)

1b. Tested on a crippled leg that was not seen as crippled during training

*ours*

model-based baseline

(no adaptation)

model-free oracle (trained only on this particular scenario + with lot more training data)

2. HALF CHEETAH - TERRAIN (Trained on random small slopes, tested on large slope)

*ours*

model-based baseline

(no adaptation)

3. HALF CHEETAH - PIER (Trained on random pier dampings)

*ours*

model-based baseline

(no adaptation)