Interactive Perception for
Deformable Object Manipulation
Submit to Conference on Humanoids 2022
Abstract: The aim of interactive perception is to use forceful interactions with the environment to bring it into a desired state. It can be applied to make the perception of the environment more effective such as to learn object properties through interaction. In this work, we address interactive perception considering deformable objects and active vision. Our approach models the regularity/structure of the integrated perception-action space by using a manifold-with-boundary formulation in a POMDP context. We develop a simulation setup to demonstrate the validity and effectiveness of the approach and perform a comparison with the state of the art methods.
Results in simulation system based on Pybullet
CubeBagClean We first initialize the camera from the side that the target rigid object is fully occluded. The ring vertices are use as SOI points. The left video corresponds to the policy learned by our proposed framework with DAVS, and the right video corresponds to the policy without DAVS.
IP with DAVS
IP without DAVS
CubeBagObst We first initialize the camera above the circular obstacle that the target rigid object is fully occluded. The ring vertices are useD as SOI points. The first video corresponds to the policy learned by our proposed framework IP with DAVS, the second video corresponds to the learned IP policy without DAVS, and the third video corresponds to the learned policy with a static camera.
IP with DAVS
IP without DAVS
Static vision
Results of failure cases
we see that the deployed policy can fail at certain camera birth places even with DAVS. But we can still observe that the camera is moving in a reasonable way. This implies that further improvements of the methods are needed to ensure better exploration in the entire action space.
which suggests that DAVS shifts the exploration-exploitation balance in the action search, improving overall performance at the cost of less exhaustive exploration.
CubeBagClean
IP with DAVS
IP without DAVS
CubeBagObst
IP with DAVS
IP without DAVS
Static vision