Sequential Manipulation Planning on Scene Graph
Ziyuan Jiao1,2, Yida Niu2, Zeyu Zhang1,2, Song-Chun Zhu1,2,3,4,5, Yixin Zhu2,3,4, Hangxin Liu2*
1 UCLA Center for Vision, Cognition, Learning, and Autonomy
2 Beijing Institute for General Artificial Intelligence
3 Institute for Artificial Intelligence, Peking University
4 School of Artificial Intelligence, Peking University
5 Department of Automation, Tsinghua University
*corresponding author
Problems: Opening the door must precede placing an object into the cabinet, and two objects must be placed in certain ways such that they do not interfere with subsequent operation, i.e., closing the door.
Solutions: We introduce predicate-like node attributes on the contact graph (i.e.,cg+) to ensure task and motion feasibility.
Abstract
We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential task planning. Augmented with predicate-like attributes, this contact graph-based representation abstracts scene layouts with succinct geometric information and valid robot-scene interactions. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with a stochastic optimization method. A task plan is then initialized by computing the Graph Editing Distance (GED) between the initial contact graphs and the goal configurations, which generates graph edit operations corresponding to possible robot actions. We finalize the task plan by imposing constraints to regulate the temporal feasibility of graph edit operations, ensuring valid task and motion correspondences. In a series of simulations and experiments, robots successfully complete complex sequential object rearrangement tasks that are difficult to specify using conventional planning language like Planning Domain Definition Language (PDDL), demonstrating the high feasibility and potential of robot sequential task planning on contact graph.
Demo
Code
To reproduce the experiments, please check:
