TacEx has been greatly improved since we presented our workshop paper.
It is easier to use, more flexible and has more features now.
Current Features Include:
GPU accelerated Tactile RGB simulation via Taxim's simulation approach
Marker Motion Simulation via FOTS
Integration of UIPC (instead of GIPC) for GPU accelerated incremental potential contact to simulate FEM soft bodies, rigid bodies, cloth, etc. in a penetration-free and robust manner
Marker Motion Simulation with FEM soft body based on the simulator used by the ManiSkill-ViTac challenge that leverages UIPC
The GPU-Taxim based simulation can be easily added to existing Isaac Lab envs. For example, some contact-rich-manipulation env's with tactile sensors.
We created a few RL environments which leverage our tactile simulation.
One of them is the Ball-Rolling-RL env.
The goal here is to move the ball to a random target position.
The speed and efficiency of the GPU-Taxim based simulation allow for (relatively) fast prototyping.
The video shows the training of a PPO agent, with proprioception + Tactile RGB images as observations. The training was done with 512 parallel env's.
We integrate the new UIPC framework to simulate the gelpad as a FEM soft body.
We can use the deformation of the FEM body to simulate the marker motion:
UIPC (libuipc) has a lot of examples. Here are some of them inside Isaac Sim.
We also have an playground to checkout/ test the tactile simulation output on the shapes from Gomes et al. For example the FOTS simulation:
This is another showcase for what you could do with the UIPC simulation. We simulate the gelpad via UIPC as a soft body and apply a GelSight-Mini texture on it.
While the deformation looks pretty nice, the image is very blurry.
This is most likely due to complex lighting and the render settings
(which we did not tweak at all for all the demonstrations here).
This showcases the UIPC-Texture simulation with lights turned off.
The blurriness seems to be significantly reduced.
rigid body with compliant contact
PhysX soft body gelpad
GIPC based soft body simulation
(sensor case turned invisible for
better view of the soft body deformation)
lifting of rigid objects
(video is partially sped up)
the object cannot be lifted
lifting of a soft object
during the grasp, the gelpads are crushed completely
the soft object is being lifted by the rigid bodies behind the gelpads
when only the gelpads are in contact, then the object falls down immeditately
gelpads were turned invisible for better view of the beam
at the end the tetrahedra mesh is shown (in white)