Additional Pouring Experiment

1) For the same scene, on which the generated robot trajectories are conditioned on, linearly perturb all three latent model axes which have been semantically aligned with the discrete label concepts above.

2) Examine the capability of the model to utilise the image input modality on which the generated robot behaviour depends. For the same specification (e.g. pour in red), different scene configurations result in different end-effector trajectories - going from pouring in red cup to pouring in blue cup for four different scene images (rows 1-4).

Change is realised by linearly interpolating the C_0 latent axis of the VAE model in the [-2, 2] range for each scene image (left to right on each row). C_0 has been optimised to discriminate between pours in different cups.

3) Given the disentangled nature of the VAE model, we generate robot trajectories conditioned on specific values for the 3 latent axes (c_0, c_1, c_2), consistent with all possible label compositions (8 possibilities). For picking the right value for each axis we use the post-training-estimated Normal distributions for each label (consult Section 3 of the paper).

On each row, the sample in the red box is a visualisation of a curated demonstration which can be described with the corresponding label composition. The following 5 samples are sampled from the optimised model.