Instance-aware segmentation of unseen objects is essential for a robotic system in an unstructured environment. Although previous works achieved encouraging results, they were limited to segmenting the only visible regions of unseen objects. For robotic manipulation in a cluttered scene, amodal perception is required to handle the occluded objects behind others. This paper addresses Unseen Object Amodal Instance Segmentation (UOAIS) to detect 1) visible masks, 2) amodal masks, and 3) occlusions on unseen object instances. For this, we propose a Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order. We evaluated our method on three benchmarks (tabletop, indoors, and bin environments) and achieved state-of-the-art (SOTA) performance.
[arXiv] [Code & Dataset] (Accepted at ICRA 2022)
All instances are unseen objects for the model. Red: occluded object, White: un-occluded object
Robot and camera setup
When the target object is occluded, grasping it directly is often infeasible due to the collision between the objects and the robot. Using UOAIS-Net, the grasping order to retrieve the target object can be easily determined. If the target object is occluded, remove the nearest unoccluded objects until the target becomes unoccluded.
UOAIS-Net Inference 1
The target object (mug) is occluded.
UOAIS-Net Inference 2
UOAIS-Net Inference 3
The target object (mug) is not occluded.
Grasping order planning using UOAIS-Net
1. Remove the nearest unoccluded object (box)
2. Remove the nearest unoccluded object (bowl)
3. Grasp the target object.
Click to see more: qualitative results and comparisons
Real RGB
UOAIS-Net (Ours)
Real RGB
UOAIS-Net (Ours)
Seunghyeok Back, Joosoon Lee, Taewon Kim, Sangjun Noh, Raeyoung Kang, Seongho Back, Kyoobin Lee
If you have any questions, please feel free to contact Seunghyeok Back: shback@gist.ac.kr