Learning Feasibility to Imitate Demonstrators with Different Dynamics

Zhangjie Cao, Yilun Hao, Mengxi Li, Dorsa Sadigh


The goal of learning from demonstration is to learn a policy for an agent (imitator) by mimicking the behavior in the demonstrations. Prior works on learning from demonstration assume that the demonstrations are collected by a demonstrator that has the same dynamics as the imitator. However, in many real-world applications, this assumption is limiting --- to improve the problem of lack of data in robotics, we would like to be able to leverage demonstrations collected from agents with different dynamics. However, this can be challenging as the demonstrations might not even be feasible for the imitator. Our insight is that we can learn a feasibility metric that captures the likelihood of a demonstration being feasible by the imitator. We develop a feasibility MDP (f-MDP) and derive the feasibility score by learning an optimal policy in the f-MDP. Our proposed feasibility measure encourages the imitator to learn from more informative demonstrations, and disregard the far from feasible demonstrations. Our experiments on four simulated environments and on a real robot show that the policy learned with our approach achieves a higher expected return than prior works. We show the videos of the real robot arm experiments here.

The red trajectories that move around the stack of books are not feasible for the 3 DoF robot arm. Imitating such trajectories may cause the 3 DoF robot arm to maximally follow these trajectories and even collide with the existing stack of books.

Therefore, it is crucial to identify trajectories that are far from feasible for the imitator, and avoid imitating them, which could lead to potential negative consequences, and instead learn more from useful demonstrations, e.g., the blue trajectories that go over the shelf that are still feasible for a robot with 3 DoF.

Real Robot Result