Learning curves of MB-MPO and four state-of-the-art model-free methods in six different Mujoco environments with a horizon of 200. MB-MPO is able to match the asymptotic performance of model-free methods with two orders of magnitude less samples.

Learning curves of MB-MPO and two MB methods in 6 different Mujoco environments with a horizon of 200 time steps. Ours achieves better asymptotic performance and faster convergence rate than previous MB methods.

Comparison of MB-MPO and model ensemble trust region policy optimization (ME-TRPO ) using 5 biased and noisy dynamic models in the HalfCheetah environment with 100 timesteps. For each dynamics model a bias is sampled uniformly from a denoted interval in every iteration. During the iterations we add Gaussian noise centered in the sampled bias with a standard deviation of 0.1 to the predictions of the dynamic model.

Comparison of our method with and without adaptation. Depicted is the development of average returns during training with three different random seeds on the half-cheetah environment with a horizon of 1000 time steps.

Hyper-parameter study in the the half-cheetah environment of a) the inner learning rate b) the number of dynamic models in the ensemble, and c) the number of meta gradient steps before collecting real environment samples and refitting the dynamic models.