TacIPC: Intersection- and Inversion-free FEM-based Elastomer Simulation For Optical Tactile Sensors

Abstract

Tactile perception stands as a critical sensory modality for human interaction with the environment. Among various tactile sensor techniques, optical sensor-based approaches have gained traction, notably for producing high-resolution tactile images. This work explores gel elastomer deformation simulation through a physics-based approach. While previous works in this direction usually adopt the explicit material point method (MPM), which has certain limitations in force simulation and rendering, we adopt the finite element method (FEM) and address the challenges in mesh penetration and element inversion with incremental potential contact (IPC) [1] method. As a result, we present a simulator named TacIPC, which can ensure numerically stable simulations while accommodating direct rendering and friction modeling. To evaluate TacIPC, we conduct four tasks: pseudo-image quality assessment, marker displacement prediction, slip-induced rotation prediction, and deformed geometry estimation. These tasks show its superior efficacy in reducing the sim-to-real gap. Our method can also seamlessly integrate with existing simulators. 

Video

Overview

Codes

We have published our source codes on GitHub. 

Results

Pseudo-image quality assessment

Marker displacement prediction

Marker displacement in rotational (a) and shearing (b) friction experiments. The red line represents the displacement vectors of markers during the rotation or shearing process, where the end with a green dot on each line represents the initial marker position.  Pixel values under the images are the average displacement of 20 selected markers which correspond to the 20 green dots in the top right sub-images of (a) and (b).

Deformed geometry estimation

We train a U-Net to estimate the depth map of the contact object given its tactile image, and then collect 20 tactile images for each of the 24 objects meshed with various random poses to obtain the training dataset by using our TacIPC simulator. 

We randomly split the simulation data samples into a training set composed of 408 images and a test set including 72 images where the test objects are included in the training dataset. We first train the network on this dataset and then validate it on real images collected from the real world to examine the sim-to-real gap between the TacIPC simulator and the real world. 

Slip-induced rotation estimation 

We train an MLP network to estimate the slip rotation angles. The training data is generated from Tacchi [2] and TacIPC simulators, each of which produces 1520 data samples. After training the rotation estimation network, we test it on data from the real world. As shown in (a), we control an AG95 gripper to grasp the test object, and we record and track the marker motion induced by the slip (the marker motion is shown in (b)). The test objects are a wooden stick and a pen, as shown in (c). 

For each object, we test for 5 runs using different contact forces to collect different slip-induced rotation angles and the corresponding MC-Tac marker displacement. We use index 1 ~ 5 to label these experiments. Estimated rotation values and ground truths are shown in the TABLE III and TABLE IV below, where the mean absolute errors of TacIPC and Tacchi [2] are 0.81 degrees and 6.91 degrees respectively, indicating the high consistency between the TacIPC simulation marker motion data with the real-world data. 

Object Classification

To further compare the sim2real gaps, we train a U-Net model to classify the test objects from pseudo tactile images on the previously described datasets (generated by TacIPC and Tacchi [2] respectively) used to train the depth estimation network. We test the trained model on a real-world dataset, as shown in TABLE I above. 

Ablation Study

In FEM-based simulation, different discretization leads to various physical accuracy and runtime. We test different tactile sensor elastomer tetrahedral mesh discretizations generated by uniform meshing and adaptive meshing methods by using them to generate tactile images and estimate contact object depth maps using these generated images.

Uniform Meshing

By uniform meshing, we mean discretizing the object into a mesh where all of its edge lengths are as close as possible to a specified constant. Below are three uniform discretized tetrahedral meshes with average edge lengths of 0.5mm, 0.375mm, 0.25mm respectively. 

Adaptive Meshing

By adaptive meshing, we mean the density of vertices reaches the maximum, in other words, the average edge length reaches the minimum, around the contact-rich region which is the central region of the elastomer front surface. In practice, we use the tetrahedral mesh discretization algorithm provided by ABAQUS to generate adaptive meshes by setting different edge length parameters for different region. The two adaptive meshes below both have an average edge length of 1mm for their back surfaces. As the figure shows, the mesh discretization on the left has a 0.25mm average edge length at the central region and a 4mm average edge length for its edge region; the mesh discretization in the right column has average edge lengths of 0.1mm, 0.15mm, and 4mm for its central, middle and edge regions respectively. 

Results

Below shows the tactile images generated by these discretized meshes and the corresponding depth maps estimated by the U-Net previously.  From the results we observe that to achieve similar tactile image quality, adaptive meshing needs far fewer vertices and cells than uniform meshing. In all other experiments, we use the adaptive mesh discretization illustrated in the most right column (highlighted by the red borders). The discretization which results in the shortest runtime was highlighed by orange borders. 

Real-world Sensors

In our experiments, we use two open-source MC-Tac as the tactile sensor. One with markers and one without markers. All the white dots in the image are the markers evenly distributed over the elastomer surface.  Few of them are highlighted by red arrows and dots below.

Parameter Settings

Baseline Tacchi [2] Paramters Setting: dt = 1e-4s, Elastomer Young's modulus: 1.23e5 Pa, Elastomer Poisson's Ratio: 0.43

For mass densities, we keep the setting in Tacchi [2] Official codes unchanged. 

Time Analysis

For the current implementation of TacIPC, the most time-consuming part is the linear system solving, which occupies ~96% of the total runtime and is currently implemented using a direct solver on the CPU. In the future, we could use a GPU-based parallel iterative solver for linear system solving to further accelerate TacIPC. A detailed breakdown pie chart is shown above.
We admit that our simulator is currently slow for real-time applications, but it could support automatic offline data generation for sim-to-real learning policy transfer. We have conducted a slip detection and estimation experiment to show such potential. In future works, we plan to use it to predict contact forces which are hard to accurately measure in the real world with low costs. 

Sensitivity Range

We conducted a large shearing experiment and a large rotation one in TacIPC to check its sensitivity range. TacIPC generates results highly consistent with the real-world ground truth within a translational range of ~2mm and a rotational range of ~0.8rad. Due to the high stiffness of our MC-Tac elastomer, larger deformation out of this range rarely happens in the real world. 

FEM-based Baseline Comparison

Ansys

Ansys is a popular FEA software, but mesh penetration are still common during contact-rich task, as shown in the figures above where a yellow object is pressed onto a blue gel elastomer. 

The supplementary material of IPC [2] also shows this phenomenon, as shown below. 

The above image is from the Supplement B: Comparison Details  of the paper Incremental Potential Contact: Intersection- and Inversion-free Large Deformation Dynamics [2]. 

References

[1] M. Li, Z. Ferguson, T. Schneider, T. R. Langlois, D. Zorin, D. Panozzo, C. Jiang, and D. M. Kaufman, “Incremental potential contact: intersection-and inversion-free, large-deformation dynamics.” ACM Trans. Graph., vol. 39, no. 4, p. 49, 2020

[2] Z. Chen, S. Zhang, S. Luo, F. Sun, and B. Fang, “Tacchi: A pluggable and low computational cost elastomer deformation simulator for optical tactile sensors,” IEEE Robotics and Automation Letters, vol. 8, no. 3, pp. 1239–1246, 2023