Factory: Fast Contact for Robotic Assembly
Presented at Robotics: Science and Systems (RSS) 2022
What is Factory?
Factory is a research project that consists of simulation methods and robot learning tools for contact-rich interactions, demonstrated for robotic assembly. The project includes
Methods for simulating contact-rich interactions, including a novel GPU-based synthesis of SDF-based collisions, contact reduction, and a Gauss-Seidel solver,
High-quality assets, robot controllers, and Gym-style learning environments for robotic assembly, and
Proof-of-concept RL policies for robots performing contact-rich tasks, namely nut-and-bolt assembly.
The methods and environments from the Factory project are available to users through NVIDIA's PhysX physics engine (since PhysX 5.1), Isaac Gym simulator (since Preview Release 4), and Isaac Sim simulator (since 2023.1.0).
The methods are also available in NVIDIA's latest robotics simulator, Isaac Lab, with environments to come in the near future.
Research Paper
Our research paper is now available on arXiv.
Please also see our follow-up papers, IndustReal and AutoMate, on sim-to-real transfer of RL policies trained with Factory.
Short Video (5 min)
Note: To view the video in fullscreen, click the title of the video to open it in Vimeo. Then click the fullscreen button on the bottom-right.
Long Video (17.5 min)
Note: To view the video in fullscreen, click the title of the video to open it in Vimeo. Then click the fullscreen button on the bottom-right.
Sneak Preview
A simulation in the pipeline, mixing rigid and deformable bodies:
Getting Started
The simulation methods and robot learning tools from Factory are now available. Here's how to get started:
Download NVIDIA Isaac Gym Previous Release 4 to get access to our simulation methods.
Familiarize yourself with Isaac Gym through the official documentation. (See the docs folder after installation.)
Clone the Isaac Gym Envs repo to get access to the Factory assets, environments, controllers, and RL policies.
Read the RL examples documentation (esp. the Factory section) and Factory documentation to start running the Factory examples.
Import your own assets, create your own contact-rich environments, and train your own RL policies.
Looking Ahead
In the near future, the Factory methods and tools will also be available within NVIDIA Isaac Lab, which is a unified platform for NVIDIA's robotics simulation tools. To get started, download NVIDIA Isaac Lab, and stay tuned for an integration of Factory-style environments.
Frequently Asked Questions
When questions are asked by collaborators and readers that may be beneficial to the research community, they will be posted here.
Q: Can Factory be used to simulate arbitrary non-convex triangular meshes of the user's choice?
A: Yes. All that is required is a URDF description of the mesh with an additional <sdf> tag; see Listing 1 in the Appendix of the paper. We have tested the simulator on a wide range of industrial-style components, as well as other complex shapes (e.g., bowls, toruses) with robust results.
Q: What are assets? Why is asset design important for physics simulation?
A: Assets are geometric and/or visual representations of an object, usually consisting of parametric CAD models, triangular surface meshes (e.g., OBJ), and/or textures. Assets for many mechanical parts can be found in online repositories, such as TurboSquid and GrabCAD.
However, these models often prioritize visual appeal, rather than exact reproduction of real-world geometry. As a result, physics simulation of these assets will often produce unrealistic results with respect to the real world. In many cases, 2 assets that are intended to mate (e.g., a plug and socket) may actually interpenetrate when one object is fully inserted into another. Without accurate assets, accurate physics simulation is extremely challenging.
We provide assets that are either 1) designed according to real-world international manufacturing standards, or 2) sourced directly from manufacturers. For the manufacturer-sourced models, we thoroughly inspect them for undesired artifacts and carefully simplify external geometry (i.e., the geometry of features that do not come into contact with features of a mating asset) for accurate and efficient simulation.
Q: What are typical clearances between real-world parts?
A: ~0.1 mm diametral clearance is a reasonable ballpark value for small parts, but clearances may vary substantially from part to part. See Table VII and Table VIII in the Appendix of the paper for the clearances between the part models that we provide and/or simulate.
Q: How much time is required to train the RL policies?
A: The majority of policies in the paper were trained for 1024 epochs in batches of 3-4 on a single NVIDIA RTX 3090 GPU. Each batch took approximately 1-1.5 hours to train in total.
Q: How did you render your simulations?
A: All photorealistic renderings in the paper and videos were generated with NVIDIA's Omniverse Create.