Meta-World ML-10 test tasks

The last rollout episode with the best return.

■ Fail ■  Success

SDVT-LW's generated virtual tasks

Redner of the last rollout episode and the visitation map for all rollout episodes.

All states are from the task "Reach".

●  Object Π  Gripper

Learned Context of all ML-10 Tasks

We report the parameters of the learned contexts  of three seeds for all tasks during a meta-episode:

• Categorical weight for subtask decomposition y 

• Mean and log std of the Gaussian distribution to sample continuous context z.

• 1 meta-episode = 10 rollout episodes 1 rollout episode = 500 steps

We demonstrate that the learned contexts mostly stay constant after some initial steps on training tasks.

How the tasks are decomposed varies over random seeds.

— Dim. 1 — Dim. 2 — Dim. 3 — Dim 4 — Dim 5

ML-45 Subtask Decomposition

We demonstrate decompositions for SDVT (K=45) and SDVT-LW (K=5)
(First random seed for both figures)

Analogous to the result on ML-10, similar tasks share subtasks.

Discrepancy between Success Rate and Return

SDVT-LW achieves high returns but low success rates on simple tasks such as Reach, Push, and Pick-place. 

This is due to the discrepancy between return and the success rates as SDVT with mean returns solve the tasks well as below.

RL2 with high variance in action succeeds by chance but with low returns.

■ Fail ■  Success