Bimanual manipulation with tactile feedback will be key to human-level robot dexterity. Here we introduce a dual-arm tactile robotic system (Bi-Touch) based on the Tactile Gym 2.0 setup that integrates two affordable industrial-level robot arms with low-cost high-resolution tactile sensors. 

We present a suite of bimanual manipulation tasks tailored towards tactile feedback: bi-pushing, bi-reorienting, and bi-gathering. To learn effective policies, we introduce appropriate reward functions for these tasks and propose a novel goal-update mechanism with deep reinforcement learning. We also apply these policies to real-world settings with a tactile sim-to-real approach. Our analysis highlights and addresses some challenges met during the sim-to-real application. Finally, we demonstrate the generalizability and robustness of this system by experimenting with different unseen objects with applied perturbations in the real world.

•   Arxiv 🔗    •    Code (New)🔗

Overview of the proposed dual-arm tactile robotic system (Bi-Touch) with sim-to-real deep RL. a) Deep RL is applied to learn policies for three simulated bimanual tactile manipulation tasks (red arrows show desired displacements) using Tactile Gym. b) Real-to-sim tactile image generator learned for the surface feature. c) The real-world evaluation feeds real tactile images through the generator into the RL policy concatenated with proprioceptive information.

The performance of our low-cost sim-to-real deep RL dual-arm tactile robot system was evaluated in these four bimanual tasks in the real world. We introduced appropriate reward functions for these tasks in simulation, then investigated how these policies apply to the real world. The experimental results show that the developed dual-arm tactile system is effective for all tasks on real objects unseen in the simulation learning.