Introduction

In this project, we focus on enabling robots to handle objects with varying tactile properties effectively in real-world scenarios, under the assumption of limited visual input. Consider the task of relocating objects with vastly different properties, like a foam sponge versus a heavy solid box: There is a chance that the robot may drop the heavy box if it's picked up with a loose grip, so a safer approach would be to drag the box instead. If the box were light, however, it could easily be lifted and moved to the desired location. Conversely, the sponge is extremely deformable and thus needs to be dragged to avoid permanent deformation.

The core of this project lies within utilizing the Axia80 F/T sensor attached to the UR5 robot arm to discern the most appropriate method for object manipulation—be it pushing or picking-and-placing—based on the object's physical characteristics.

With the help of an external camera and AR tags, the robot knows the vague locations of the objects and the desired position. The robot does not have any other information about the object and must determine how to manipulate the object after testing the object for characteristics such as deformability and weight.