Changkyu Song and Abdeslam Boularias,

Inferring 3D Shapes of Unknown Rigid Objects in Clutter through Inverse Physics Reasoning,

IEEE Robotics and Automation Letters (RA-L 2018) and presented in IEEE International Conference on Robotics and Automation (ICRA 2019), May 20-24th, 2019, Montreal, Canada. [PDF] [Poster] [Video] [Webpage] [Article in Tech Xplore]

We present a probabilistic approach for building, on the fly, 3-D models of unknown objects while being manipulated by a robot. We specifically consider manipulation tasks in piles of clutter that contain previously unseen objects. Most manipulation algorithms for performing such tasks require known geometric models of the objects in order to grasp or rearrange them robustly. One of the novel aspects of this work is the utilization of a physics engine for verifying hypothesized geometries in simulation. The evidence provided by physics simulations is used in a probabilistic framework that accounts for the fact that mechanical properties of the objects are uncertain. We present an efficient algorithm for inferring occluded parts of objects based on their observed motions and mutual interactions. Experiments using a robot show that this approach is efficient for constructing physically realistic 3-D models, which can be useful for manipulation planning. Experiments also show that the proposed approach significantly outperforms alternative approaches in terms of shape accuracy.

Dataset

Our dataset with YCB objects includes the tabletop scenes as well as piles of objects inside a tight box that can be seen in the attached video. Objects in this dataset are severely occluded. We split the dataset into two subsets, one with only static scenes and another with only dynamic ones. Static scenes are 12 in total. Dynamic scenes, 13 in total, include at least one robotic pushing action per scene. The dataset contains the color and depth images as well as our segmentation results and their ground truth volume occupancy. You can download the dataset [HERE].

The list of YCB objects used in dataset is following.

002_master_chef_can, 003_cracker_box, 004_sugar_box, 005_tomato_soup_can, 006_mustard_bottle, 008_pudding_box, 009_gelatin_box, 010_potted_meat_can, 012_strawberry, 013_apple, 014_lemon, 015_peach, 021_bleach_cleanser, 036_wood_block, 053_mini_soccer_ball, 054_softball, 055_baseball, 056_tennis_ball, 071_nine_hole_peg_test, 077_rubiks_cube

Results