Imitation Learning (IL) is a promising paradigm for learning dynamic manipulation of deformable objects since it does not depend on difficult-to-create accurate simulations of such objects. However, the translation of motions demonstrated by a human to a robot is a challenge for IL, due to differences in the embodiments and the robot’s physical limits. These limits are especially relevant in dynamic manipulation where high velocities and accelerations are typical.
To address this problem, we propose a framework that first maps a dynamic demonstration into a motion that respects the robot’s constraints using a constrained Dynamic Movement Primitive. Second, the resulting object state is further optimized by quasi-static refinement motions to optimize task performance metrics. This allows both efficiently altering the object state by dynamic motions and stable small-scale refinements. We evaluate the framework in the challenging task of bag opening, designing the system BILBO: Bimanual dynamic manipulation using Imitation Learning for Bag Opening. Our results show that BILBO can successfully open a wide range of crumpled bags, using a demonstration with a single bag.
Our framework combines a learned dynamic motion primitive that adheres to the robots' constraints, for quick task progression, with quasi-static motions for final adjustments.
In the task of bag-opening, BILBO expands the rim area and volume using the dynamic motion. Once sufficiently high values are reached, the rim roundness (elongation) is refined using a quasi-static motion. The dynamic primitive is learned from a single human demonstration using a constrained DMP, which generalizes to a wide range of bags.
We evaluate BILBO on five different bags with the following properties:
Bag A: biodegradable plastic, 38 cm x 40 cm
Bag B: polyethylene, 41 cm x 34 cm
Bag C: polyethylene, the rim contains a drawstring, 42 cm x 49 cm
Bag D: polyethylene, 49 cm x 41 cm
Bag E: polyethylene, the rim contains a drawstring, 53 cm x 59 cm
In the video BILBO is compared against a constrained DMP baseline, TC-DMP:
BILBO: Encodes the demonstrated dynamic fling using the constrained DMP from (https://doi.org/10.1007/s10514-022-10067-4) and uses a linear motion for refining the rim roundness.
TC-DMP: Replaces the constrained DMP in BILBO with the method from (https://doi.org/10.1109/LRA.2021.3058874).
The paper is based on the research done in Eric Hannus' Master's thesis (available at https://aaltodoc.aalto.fi/items/05333c14-4fa7-46f7-bcbe-7954b0754aa6). See this document for more details about, for example, processing of the bag state and experiment practicalities.
Below are some example videos at original speed of ideal runs which open the bag with a single or a only a few dynamic flings.
Eric Hannus
School of Electrical Engineering
Aalto University, Finland
To cite this work, please use the following BibTex entry:
@inproceedings{hannus2024dynamic,
title={Dynamic Manipulation of Deformable Objects using Imitation Learning with Adaptation to Hardware Constraints},
author={Eric Hannus and Tran Nguyen Le and David Blanco-Mulero and Ville Kyrki},
year={2024},
journal = {arXiv preprint arXiv:2403.12685},
eprint={2403.12685},
archivePrefix={arXiv},
primaryClass={cs.RO}
}