Abstract

We consider the problem of building an assistant robot that can help humans in everyday house-hold cleanup tasks. The creation of such an autonomous system in real-world environments is inherently a challenging multi-objective task, comprising of -- (i) Detection of objects out-of-place, and prediction of potentially correct placements, (ii) Fine-grained manipulation for stable object grasping, and (iii) Room-to-room navigation for transferring objects in unseen environments. In this work, we systematically tackle each component, and integrate them into a complete object rearrangement pipeline. To validate our proposed system, we conduct multiple experiments on a real robotic platform involving both long-horizon, and complex pick-and-place tasks.