Figure1. (A) Direct learning long-horizon action trajectories of geometric assembly may face many challenges: grasping ungraspable points, grasping points not suitable for assembly (e.g., seams of fragments), robot colliding with parts and the other robot. (B) We formulate this task into 3 steps: pick-up, alignment and assembly. For assembly, we predict the direction that will not result in part collisions. For alignment, we transformed any assembled poses to poses easy for robot to manipulate from the initial poses without collisions. For pick-up, we learn point-level affordance aware of graspness and the following 2 steps. (C) Real-World Evaluations with affordance predictions on two mugs and the corresponding manipulation.
Abstract
Shape assembly, the process of combining parts into a complete whole, is a crucial skill for robots with broad real-world applications. Among the various assembly tasks, geometric assembly—where broken parts are reassembled into their original form (e.g., reconstructing a shattered bowl)—is particularly challenging. This requires the robot to recognize geometric cues for grasping, assembly, and subsequent bimanual collaborative manipulation in varied fragments. In this paper, we exploit the geometric generalization capability of point-level affordance, learning affordance that enables both generalization and collaboration in long-horizon geometric assembly tasks. To address the diversity in geometries of broken parts for evaluation, we also introduce a real-world benchmark featuring geometric variety and global reproducibility. Extensive experiments demonstrate the superiority of our approach over both previous affordance-based and imitation-based methods.
Video Presentation
video-ID1479-1080p.mp4Framework
Figure 2. BiAssembly Framework Overview. With the point cloud observation and Imaginary Assembled Shape, the model predicts the disassembly direction in which the disassembled part poses can be easily reached by manipulating the raw parts under the guidance Bi-Affordance.
Real-World Benchmark
Figure 3. Part A illustrates the pipeline for scanning and reconstructing real objects. Part B presents examples of fractured parts from various categories, showcasing diverse geometries.
Experiments in Simulation
Figure 4. We present qualitative results of predicted affordance maps and robot actions from our method. In each row, from left to right, we respectively present the input observation, the predicted affordance maps for the two fractured parts, and the bimanual actions for the pick-up, alignment, and assembly steps. In the top part are novel shapes from training categories, while in the bottom part are shapes from unseen categories.
Experiments in Real World
Affordance
Manipulation
Mug
Mug.mp4Plate
Plate.mp4TeaCup
TeaCup.mp4WineGlass
WineBottle.mp4Bowl
Bowl.mp4BeerBottle
BeerBottle.mp4