Logic Learning from Demonstrations for Multi-step Manipulation Tasks in Dynamic Environments

Abstract: 

Learning from Demonstration (LfD) stands as an efficient framework for imparting human-like skills to robots. Nevertheless, designing an LfD framework capable of seamlessly imitating, generalizing, and reacting to disturbances for long-horizon manipulation tasks in dynamic environments remains a challenge. To tackle this challenge, we present Logic-LfD, which combines Task and Motion Planning (TAMP) with an optimal control formulation of Dynamic Movement Primitives (DMP), allowing us to incorporate motion-level via-point specifications and to handle task-level variations or disturbances in dynamic environments. We conduct a comparative analysis of our proposed approach against several baselines, evaluating its generalization ability and reactivity across three long-horizon manipulation tasks. Our experiment demonstrates the fast generalization and reactivity of Logic-LfD for handling task-level variants and disturbances in long-horizon manipulation tasks.

Experiment Results:

In the following, we illustrate videos of reactive task and motion planning with Logic-LfD for solving tasks under various levels of disturbances. 

Level 1: Motion-level Variant/Disturbance: Blocks are subjected to disturbances in different positions, while logical states remain consistent with the original or expected conditions;

Level 2: Slight Task-level Variant/Disturbance: Blocks undergo disturbances, resulting in a different logical state. This logical state aligns with that seen in the demonstration for the template task;

Level 3: Severe Task-level Variant/Disturbance: Blocks are disturbed to a novel and previously unseen logical state in the demonstration;

Level 4: Extreme Task-level Variant/Disturbance: A new block is introduced into the scene that significantly influences the execution of the initial plan to achieve the target goal.

Experiment 1: Block Stacking 

Experiment 2: Workspace Reach

Experiment 3: Dual-arm Transportation 

Experiment 4: Real-world Block Stacking

Citation:

@article{zhang2024logic,

  title={Logic Learning from Demonstrations for Multi-step Manipulation Tasks in Dynamic Environments},

  author={Zhang, Yan and Xue, Teng and Razmjoo, Amirreza and Calinon, Sylvain},

  journal={IEEE Robotics and Automation Letters},

year={2024},

  volume={9},

  number={8},

  pages={7214-7221},

 doi={10.1109/LRA.2024.3418276}

}