TacMMs: Tactile Mobile Manipulators for Warehouse Automation

Zhuochao He, Xuyang Zhang, Simon Jones, Sabine Hauert, Dandan Zhang, Nathan. F. Lepora

Email: sq21457@alumni.bristol.ac.uk---------Zhuochao He, zhangxuyang0223@gmail.com--------Xuyang Zhang,

simon2.jones@brl.ac.uk-------- Simon Jones, sabine.hauert@bristol.ac.uk--------Sabine Hauert,

ye21623@bristol.ac.uk----------Dandan Zhang, n.lepora@bristol.ac.uk-----------Nathan.F.Lepora

Abstract

Multi-robot platforms are playing an increasingly important role in warehouse automation for efficient goods transport. This paper proposes a novel customization of a multi-robot system to become Tactile Mobile Manipulators (TacMMs). Each TacMM integrates a soft optical tactile sensor and a mobile robot with a load-lifting mechanism, to enable cooperative trans portation on tasks requiring coordinated physical interaction. More specifically, we mount the TacTip (biomimetic optical tactile sensor) on the Distributed Organisation and Transport System (DOTS) mobile robot. Then the tactile information helps the mobile robots adjust the relative robot-object pose to increase the efficiency of load-lifting tasks. This study compares the performance of using two TacMMs with tactile perception with traditional vision-based pose adjustment for load-lifting. The results show that the average success rate of the TacMMs (66%) is improved over a purely visual-based method (34%), with a larger improvement when the mass of the load was non-uniformly distributed. This initial study validates the viability of the TacMM concept with two robots, and we expect greater benefits when multiple TacMMs work together, analogous to how multiple fingered robot hands have greater in-hand dexterity than pinch grippers.

Video

Demonstration Video of Lifting (TacMMs and Baseline)

Hardware

TacMM system comprises three parts: the DOTS mobile robot; the mounted TacTip tactile sensor and their connecting base. The DOTS mobile robot is equipped with an omnidirectional movement mechanism, a lifting platform with 2kg maximum payload and four cameras for a local vision system. The TacTip is mounted using a connection base on top and oriented to the front of the mobile robot. A TacTip is composed of a built-in camera, a mounting base, a LED ring, a lens and a 3D-printed skin within internal pins. When the TacTip contacts an object, the deformation of the skin causes the pins to lever which amplifies the positional deviation of the markers captured by the internal camera. Here, the tactile images are pre-processed and input into aPoseNet, a CNN-based pose estimation neural network to predict the relative sensor object contact pose, including contact orientation and depth.

System Architecture

Each mobile robot in the TacMM system is based on ROS2, which processes the sensor data, controls the robot motion and implements communications between robots. The tactile-based system adjusts the relative pose between the mobile robot and the object sequentially over multiple contacts. In the real vision system, the relative pose is detected using ArUco markers. The robot then adjusts its pose until the central line of the TacTip end effector is normal to the surface of the target object. The TacTip deformation upon contact is captured by an internal camera and the tactile images are used as an input into the PoseNet neural network. The PoseNet predicts the contact depth and the relative angle between the normal of the object and the TacTip. After adjusting their poses, the two TacMMs must cooperate to lift the object.

Control Strategies

TacMM lifting strategy

Vision-based lifting strategy

Results

TaMM

Baseline

Empty box Two stacked boxes 200g weight on the 200g weight on the 500g weight on the bottom of the empty box top of the empty box top of the empty box

To complete the lifting task, the TacMM pose adjustment strategy is used to adjust the robots’ poses relative to the object to be lifted. TheTacMM system showed modest improvement of a purely vision-based system for the pose adjustment alone, but this was more pronounced for the load lifting task, especially as the weight of the load increases because of the capability to precisely control the contact depth.

Page updated

Google Sites

Report abuse