The first result we present is a dynamic visualization of individual skills in a given source domain, and their respective translations to a given target domain. We present these results across several source-target domain pairs. This result serves to visualize the semantic correspondences learnt by our translation model.

We first present individual skills and their translations from the Franka Panda Robot (the source domain) to the Sawyer Robot (the target domain).

In each column, we present a skill on the Franka Panda robot (in the top row), and the corresponding translated skill on the Sawyer robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We also present individual skills and their translations from the Baxter Left Hand (the source domain) to the Franka (the target domain).

In each column, we present a skill on the Left Hand of the Baxter robot (in the top row), and the corresponding translated skill on the Franka robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We also present individual skills and their translations from the Baxter Right Hand (the source domain) to the Franka (the target domain).

In each column, we present a skill on the Right Hand of the Baxter robot (in the top row), and the corresponding translated skill on the Franka robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We now present individual skills and their translations from the Baxter Right Hand (the source domain) to the Baxter Left Hand (the target domain).

In each column, we present a skill on the Right Hand of the Baxter robot (in the top row), and the corresponding translated skill on the Left Hand of the Baxter robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We also present individual skills and their translations from the Baxter Left Hand (the source domain) to the Sawyer (the target domain).

In each column, we present a skill on the Left Hand of the Baxter robot (in the top row), and the corresponding translated skill on the Sawyer robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We also present individual skills and their translations from the Baxter Right Hand (the source domain) to the Sawyer (the target domain).

In each column, we present a skill on the Right Hand of the Baxter robot (in the top row), and the corresponding translated skill on the Sawyer robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

In order to successfully translate from human demonstrators to robots, the first component is to translate human skills to robot skills. While the above visualizations show the ability of our approach to translate individual skills across various robots, we now present results of translating such individual skills from human demonstrators to each of the 4 robots.

We first present individual skills and their translations from Human demonstrators (the source domain) to the Franka Panda Robot (the target domain).

In each column, we present a skill of a human demonstrator (in the top row), and the corresponding translated skill on the Franka Panda robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We first present individual skills and their translations from Human demonstrators (the source domain) to the Baxter Left Hand (the target domain).

In each column, we present a skill of a human demonstrator (in the top row), and the corresponding translated skill on the Baxter Left Hand (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We first present individual skills and their translations from Human demonstrators (the source domain) to the Sawyer Robot (the target domain).

In each column, we present a skill of a human demonstrator (in the top row), and the corresponding translated skill on the Sawyer robot (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

We first present individual skills and their translations from Human demonstrators (the source domain) to the Baxter Right Hand (the target domain).

In each column, we present a skill of a human demonstrator (in the top row), and the corresponding translated skill on the Baxter Right Hand (in the bottom row). The skills are semantically labelled with the classical manipulation skill they most resemble. Despite the translation being learned in a purely unsupervised manner, notice the close semantic correspondence between skills in the source domain and the target domain.

The next result we present is a dynamic visualization of entire trajectories (i.e., a sequence of skills) in a given source domain, and their respective translations to a given target domain. We present these results across several source-target domain pairs. This result serves to visualize how well the learned translation model can map an entire sequence of skills from the source domain to the target domain.

We also present entire trajectories and their translations from the Franka Panda robot (the source domain) to the Sawyer robot (the target domain). In each row, we present the trajectory from the Franka Panda robot (on the left), and the corresponding translated sequence of skills / trajectory on the Sawyer robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the Franka Panda reaching in to the right of the workspace, performing a picking / grasping skill, and finally a placing skill to the front right portion of the workspace. The Sawyer robot correspondingly reaches to the left of the workspace, performs a grasping skill, and finally also executes a placing skill to the front right of the workspace.

We also present entire trajectories and their translations from the Baxter Right Hand (the source domain) to the Franka Panda robot (the target domain).

In each row, we present the trajectory from the Baxter Right Hand robot (on the left), and the corresponding translated sequence of skills / trajectory on the Franka Panda robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the Baxter Right Hand reaching in to the workspace, and subsequently performing a lifting skill. The translated trajectory on the Franka robot also correspondingly reaches into the workspace, and performs a lifting skill (albeit with a different arm shape).

We also present entire trajectories and their translations from the Baxter Right Hand (the source domain) to the Baxter Left Hand (the target domain).

In each row, we present the trajectory from the Baxter Right Hand robot (on the left), and the corresponding translated sequence of skills / trajectory on the Baxter Left Hand robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the Baxter Right Hand reaching in to the workspace, performing a lifting skill, and subsequently returning to its rest configuration. The translated trajectory on the Baxter Left Hand also correspondingly reaches into the workspace (albeit with a different arm shape), performs a small lifting skill, and also finally returns to its rest configuration.

We also present entire trajectories and their translations from the Baxter Left Hand (the source domain) to the Sawyer (the target domain).

In each row, we present the trajectory from the Baxter Left Hand (on the left), and the corresponding translated sequence of skills / trajectory on the Sawyer robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the Baxter Left Hand reaching in to its right of the workspace, performing a small placing skill to its left, and finally returning to its rest configuration. The translated trajectory on the Sawyer Robot also reaches in to the right of its workspace, and then performs a placing skill to the middle of the workspace.

We also present entire trajectories and their translations from the Baxter Right Hand (the source domain) to the Sawyer (the target domain).

In each row, we present the trajectory from the Baxter Right Hand (on the left), and the corresponding translated sequence of skills / trajectory on the Sawyer robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the Baxter Right Hand reaching into its workspace, performing a twisting skill to open the lid of a jar, and placing the lid of the jar to the side. The translated trajectory on the Sawyer robot also correspondingly reaches into the workspace, performs a twisting skill, and finally performs a placing skill to the right.

We also present entire trajectories and their translations from a Human Demonstration (the source domain) to the Baxter Left Hand (the target domain).

In each row, we present the trajectory from the Human demonstrator (on the left) and the corresponding translated sequence of skills / trajectory on the Baxter Left Hand (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the human demonstrator reaching towards the workspace, and using a rubber stamp (depicted in yellow) to mime stamping locations on the table, before finally returning to their default pose. The translated trajectory also similarly reaches towards the workspace, performs a to and fro motion similar to the stamping motion, and finally returns to its default configuration.

We also present entire trajectories and their translations from a Human Demonstration (the source domain) to the Baxter Right Hand (the target domain).

In each row, we present the trajectory from the Human demonstrator (on the left) and the corresponding translated sequence of skills / trajectory on the Baxter Right Hand (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the human demonstrator reaching towards the workspace, lifting up a toy pyramid (depicted in yellow), before finally placing it and returning to their default pose. The translated trajectory also similarly reaches towards the workspace, performing a lifting skill in the center of the workspace, before finally returning to its default configuration.

We also present entire trajectories and their translations from a Human Demonstration (the source domain) to the Franka Panda Robot (the target domain).

In each row, we present the trajectory from the Human demonstrator (on the left) and the corresponding translated sequence of skills / trajectory on the Franka Panda robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the human demonstrator reaching towards the workspace, and using a rubber stamp (depicted in yellow) to mime stamping locations on the table, before finally returning to their default pose. The translated trajectory also similarly reaches towards the workspace, perform several small to and fro motions similar to the stamping motion, and finally returns to its default configuration.

We also present entire trajectories and their translations from a Human Demonstration (the source domain) to the Sawyer Robot (the target domain).

In each row, we present the trajectory from the Human demonstrator (on the left) and the corresponding translated sequence of skills / trajectory on the Sawyer robot (on the right). A brief description of the sequence of skills executed are provided for each trajectory.

Trajectory #1: The original trajectory consists of the human demonstrator reaching towards the workspace, and grasping and moving a toy train (depicted in yellow) above a table to their left. The translated trajectory also similarly reaches to the center of the workspace, and subsequently performs a placing skill to its left.