This section compares our synchronous and asynchronous implementation of ME-PPO in four Mujoco environments. Wall-clock time corresponding to the checkpoint time-steps are reported in the table below each video.

HalfCheetah

Ant

Hopper

Walker2D