Satoshi Kataoka, Youngseog Chung, Seyed Kamyar Seyed Ghasemipour,
Pannag Sanketi, Shixiang Shane Gu, Igor Mordatch
Most successes in robotic manipulation have been restricted to single-arm gripper robots, whose low dexterity limits the range of solvable tasks to pick-and-place, insertion, and object rearrangement. More complex tasks such as assembly require dual and multi-arm platforms, but entail a suite of unique challenges such as bi-arm coordination and collision avoidance, robust grasping, and long-horizon planning. In this work we investigate the feasibility of training deep reinforcement learning (RL) policies in simulation and transferring them to the real world (Sim2Real) as a generic methodology for obtaining performant controllers for real-world bi-manual robotic manipulation tasks. As a testbed for bi-manual manipulation, we develop the “U-Shape Magnetic Block Assembly Task”, wherein two robots with parallel grippers must connect 3 magnetic blocks to form a “U”-shape. Without a manually-designed controller nor human demonstrations, we demonstrate that with careful Sim2Real considerations, our policies trained with RL in simulation enable two xArm6 robots to solve the U-shape assembly task with a success rate of above 90% in simulation, and 50% on real hardware without any additional real-world fine-tuning. Through careful ablations, we highlight how each component of the system is critical for such simple and successful policy learning and transfer, including task specification, learning algorithm, direct joint-space control, behavior constraints, perception and actuation noises, action delays and action interpolation. Our results present a significant step forward for bi-arm capability on real hardware, and we hope our system can inspire future research on deep RL and Sim2Real transfer of bi-manual policies, drastically scaling up the capability of real-world robot manipulators.
To study the efficacy of simulated deep RL and Sim2Real transfer, we design the “U-Shape Magnetic Block Assembly Task” as a testbed for studying key challenges of bi-manual manipulation in the real world. In this task, two robot arms equipped with parallel grippers must connect 3 magnetic blocks in order to form a ”U” shape.
Concurrent to this work, we have been studying bimanual assembly via blueprint assembly environments with simulated direct actuations. To briefly introduce that research, it demonstrated training of a single agent that can simultaneously solve all seen and unseen assembly tasks via a combination of large-scale RL, structured policies, and multi-task training. Our current efforts in this directions can be viewed in the link below, with a video of our results presented below.