Embedded Files
Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling
Instance-aware segmentation of unseen objects is essential for a robotic system in an unstructured environment. Although previous works achieved encouraging results, they were limited to segmenting the only visible regions of unseen objects. For robotic manipulation in a cluttered scene, amodal perception is required to handle the occluded objects behind others. This paper addresses Unseen Object Amodal Instance Segmentation (UOAIS) to detect 1) visible masks, 2) amodal masks, and 3) occlusions on unseen object instances. For this, we propose a Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order. We evaluated our method on three benchmarks (tabletop, indoors, and bin environments) and achieved state-of-the-art (SOTA) performance.
Paper & Code
Paper & Code
[arXiv] [Code & Dataset] (Accepted at ICRA 2022)
Video
Video
UOAIS-Net Demonstrations
UOAIS-Net Demonstrations
All instances are unseen objects for the model. Red: occluded object, White: un-occluded object
Robotic Application: Occluded Object Retrieval
Robotic Application: Occluded Object Retrieval
Robot and camera setup
When the target object is occluded, grasping it directly is often infeasible due to the collision between the objects and the robot. Using UOAIS-Net, the grasping order to retrieve the target object can be easily determined. If the target object is occluded, remove the nearest unoccluded objects until the target becomes unoccluded.
UOAIS-Net Inference 1
The target object (mug) is occluded.
UOAIS-Net Inference 2
UOAIS-Net Inference 3
The target object (mug) is not occluded.
Grasping order planning using UOAIS-Net
1. Remove the nearest unoccluded object (box)
2. Remove the nearest unoccluded object (bowl)
3. Grasp the target object.
Results and Comparisons on Benchmarks
Results and Comparisons on Benchmarks
Click to see more: qualitative results and comparisons
Real RGB
UOAIS-Net (Ours)
Real RGB
UOAIS-Net (Ours)
Author
Author
Seunghyeok Back, Joosoon Lee, Taewon Kim, Sangjun Noh, Raeyoung Kang, Seongho Back, Kyoobin Lee
If you have any questions, please feel free to contact Seunghyeok Back: shback@gist.ac.kr
Citation
@misc{back2021unseen, title={Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling}, author={Seunghyeok Back and Joosoon Lee and Taewon Kim and Sangjun Noh and Raeyoung Kang and Seongho Bak and Kyoobin Lee}, year={2021}, eprint={2109.11103}, archivePrefix={arXiv}, primaryClass={cs.RO}}Citation
Page updated
Google Sites
Report abuse