This section compares our synchronous and asynchronous implementations of ME-PPO, ME-TRPO, and MB-MPO and each of their respective performance in completing three tasks: Reaching, Shape Matching, and Lego Stacking. Further details can be found in the paper.