Hai Nguyen*, Tadashi Kozuno^, Cristian C. Beltran-Hernandez^, Masashi Hamaya^
^ OMRON SINIC X (OSX), Tokyo, Japan
* Northeastern University, Boston, USA (Internship at OSX)
This study tackles the representative yet challenging contact-rich peg-in-hole task of robotic assembly, using a soft wrist that can operate more safely and tolerate lower-frequency control signals than a rigid one. Previous studies often use a fully observable formulation, requiring external setups or estimators for the peg-to-hole pose. In contrast, we use a partially observable formulation and deep reinforcement learning from demonstrations to learn a memory-based agent that acts purely on haptic and proprioceptive signals. Moreover, previous works do not incorporate potential domain symmetry and thus must search for solutions in a bigger space. Instead, we propose to leverage the symmetry for sample efficiency by augmenting the training data and constructing auxiliary losses to force the agent to adhere to the symmetry. Results in simulation with five different symmetric peg shapes show that our proposed agent can be comparable to or even outperform a state-based agent. In particular, the sample efficiency also allows us to learn directly on the real robot within 3 hours.
This study proposes to leverage the potential symmetry of the task for learning a memory-based actor-critic agent in a sample-efficient manner. Specifically, the peg and the hole (as well as many industrial parts) are often symmetric or highly similar on multiple sides for easy assembly. For instance, Fig. 1 depicts six initial peg-to-hole positions, including two reflected and four rotational positions. These positions are equivalent due to the symmetry of a round hole. Consequently, one can transform (rotate or reflect) one trajectory initiated in one position to create valid trajectories in the other positions, indicating the possibility for data augmentation. Moreover, because these trajectories are equivalent, a critic should evaluate them the same.
