Efficient Object Rearrangement via Multi-view Fusion
Dehao Huang1,2, Chao Tang1,2, Hong Zhang1,2
[1] Shenzhen Key Laboratory of Robotics and Computer Vision, SUSTech, Shenzhen, China.
[2] Department of Electronic and Electrical Engineering, SUSTech, Shenzhen, China.
Abstract: The prospect of assistive robots aiding in object organization has always been compelling. In an image-goal setting, the robot rearranges the current scene to match the single image captured from the goal scene. The key to an image-goal rearrangement system is estimating the desired placement pose of each object based on the single goal image and observations from the current scene. In order to establish sufficient associations for accurate estimation, the system should observe an object from a viewpoint similar to that in the goal image. Existing image-goal rearrangement systems, due to their reliance on a fixed viewpoint for perception, often require redundant manipulations to randomly adjust an object's pose for a better perspective. Addressing this inefficiency, we introduce a novel object rearrangement system that employs multi-view fusion. By observing the current scene from multiple viewpoints before manipulating objects, our approach can estimate a more accurate pose without redundant manipulation times. A standard visual localization pipeline at the object level is developed to capitalize on the advantages of multi-view observations. Simulation results demonstrate that the efficiency of our system outperforms existing single-view systems. The effectiveness of our system is further validated in a physical experiment.
Overview:
(a) depicts the setting for the image-goal object rearrangement task. (b) shows the advantages of multi-view compared to single-view.
Supplementary Video:
Pipeline:
Overview of the perception module
Overview of the planning module
Qualitative Results:
Qualitative results: Each row is a visualization of the intermediate results for an object in an experimental scene. The candidate is the top retrieved object region within the database. In the figure of the relative pose, the arrow points from the initial object coordinate system of the current scene to the estimated object pose of the goal image. The last column shows the feasible grasping pose and its corresponding placement pose for the current object.
Authors:
Citation:
@article{huang2023efficient,
title={Efficient Object Rearrangement via Multi-view Fusion},
author={Huang, Dehao and Tang, Chao and Zhang, Hong},
journal={arXiv preprint arXiv:2309.08994},
year={2023}
}
