Motivation:

Many daily manipulation tasks, such as cooking and furniture assembly, require robots to sequentially interact with multiple objects over long planning horizons. To complete such tasks efficiently, a robot must reason about which objects are relevant to the task, when they should be manipulated, and which intermediate states can serve as useful milestones. Inspired by how humans use recipes or assembly manuals to decompose complex tasks, this paper asks: 

• Can robots learn to summarize long-horizon task instructions from past experience? 

• Which intermediate states should be considered subgoals? 

• How can these subgoals be discovered from demonstrations? 

• How should the robot select the most relevant subgoal during online planning? 

• Which objects should the planner attend to when moving between consecutive subgoals?

Key Ideas:

We propose Learn2Decompose, a framework that answers these questions through three key submodules: 

1) Subgoal Discovery defines subgoals as intermediate states that the system must pass through to reach the task goal, and summarizes them from past experiences by formulating the problem as a Sequential Pattern Mining process; 

2) Distance Learning learns a multi-object state distance by training a Graph Neural Network (GNN) on the same past experiences, allowing the robot to select the closest subgoal during online replanning;

3) Object Attention identifies the objects that are important for transitioning between consecutive subgoals, allowing the planner to ignore irrelevant objects during replanning and thereby improve planning efficiency.

In the following, we provide a supplementary video for our project, including key robot experiment videos. Please refer to our arXiv paper for the methods and experiments of our project. In this version, we also provided additional experimental results and evaluations of our method, and a detailed discussion of the project’s limitations and future work.

Citation

@article{Zhang25RAL,

author={Zhang, Y. and Xue, T. and Razmjoo, A. and Calinon, S.},

title={Learning Problem Decomposition for Efficient Sequential Multi-object Manipulation Planning},

journal={{IEEE} Robotics and Automation Letters ({RA-L})},

year={2025},

volume={10}, 

number={12},

pages={13367--13374},

doi={10.1109/LRA.2025.3630032}

}

Contact

Feel free to contact me via: yaanzhang@outlook.com, if you have any questions!