Modulization of Bin-Picking Intelligence

Key skills:

SSH / ROS / Docker / DockerFile / Dimension Reduction / Pytorch

Motivation

The learning-based approach has greatly improved the bin picking accuracies of the robot. However, since implementing AI models for robotic task requires great amount of computing power, and this can be a great burden for robots which does not have enough computing capacity.

AI from the other Machine

We decided to develop a bin-picking intelligence module from a server computer, and communicate with the client that is at charge of controlling robot manipulators.

We used SoTA bin-picking network (GQ-CNN) from AUTOLAB [1] to make a grasp planner in a server computer, and connect with the client with ssh-ROS communication.

The client PC consists of segmentation module, MoveIt motion planner, and main module that controls the overall process.

When the segmentation module segments objects from the background scene, the main modules samples grasp candidate, and grasp planner from the server PC ranks the grasp quality of the candidates. Eventually, the robot grasps the objects with the grasp of the best quality.

Fig 1. Overall architecture of grasp planning and execution setting

Wrap it through DockerFile

The grasp planner module itself is relatively heavy, therefore we decided to make a dockerfile for the grasp planner, and make it easily downloadable.

The dockerfile of the grasp planner consists of diverse elements (ROS, tensorlfow, GQ-CNN, open-ssh), and easily be converted to a docker image and container.

Once converted, the docker container then can fully function as a grasp planner.

The dockerfile can be downloaded through this link.

Fig 2. DockerFile architecture of a grasp planner (GQ-CNN)

Fig 3. Architecture of GQ-CNN and dimension reduction for visualization. Fully-connected layer (128 dim) was reduced to 3 dimensions using PCA and t-SNE.

Fig 4. Visualization of pre-trained network's feature space (reduced in 3D)

Network Training and Visualization

We trained the network by rebuilding the network using Pytorch and train dataset given by AUTOLAB.

Moreover, to observe how much the network is trained, we made a vizualization module, reducing the feature space of the network (FC_4 in Fig 3), from 128 dimensions to 3 dimensions using PCA (Principal Component Analysis) and t-SNE method.

Relevant github repositiory can be accessed through this link.

Result and Demonstration

The bin-picking setting consists of IRB 120 robot arm with Robotiq-2f gripper, and Indy7 robot arm with Robotiq suction vaccum gripper. The detailed mechanism of the grasp planner can be observed in mahler et al [1].

References

[1] Mahler, J., Matl, M., Satish, V., Danielczuk, M., DeRose, B., McKinley, S., & Goldberg, K. (2019). Learning ambidextrous robot grasping policies. Science Robotics, 4(26), eaau4984.