Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor

Sequential pulling policies to flatten and smooth fabrics have applications from surgery to manufacturing to home tasks such as bed making and folding clothes. Due to the complexity of fabric states and dynamics, we apply deep imitation learning to learn policies that, given color (RGB), depth (D), or combined color-depth (RGBD) images of a rectangular fabric sample, estimate pick points and pull vectors to spread the fabric to maximize coverage. To generate data, we develop a fabric simulator and an algorithmic supervisor that has access to complete state information. We train policies in simulation using domain randomization and dataset aggregation (DAgger) on three tiers of difficulty in the initial randomized configuration. We present results comparing five baseline policies to learned policies and report systematic comparisons of RGB vs D vs RGBD images as inputs. In simulation, learned policies achieve comparable or superior performance to analytic baselines. In 180 physical experiments with the da Vinci Research Kit (dVRK) surgical robot, RGBD policies trained in simulation attain coverage of 83% to 95% depending on difficulty tier, suggesting that effective fabric smoothing policies can be learned from an algorithmic supervisor and that depth sensing is a valuable addition to color alone.

The code for the project is located across these repositories:

• • gym-cloth: https://github.com/DanielTakeshi/gym-cloth This code creates an OpenAI gym-style environment for fabric smoothing. We also use it to generate offline demonstrator data and to benchmark performance of analytic baseline policies.

  • baselines-fork: https://github.com/DanielTakeshi/baselines-fork This code is used for the DAgger implementation, and is built on top of OpenAI baselines to make use of parallel environments for faster training.

  • dvrk_python: https://github.com/DanielTakeshi/dvrk-python This code is used for the physical surgical robot experiments.

The repositories have a fair amount of documentation, but there are a lot of moving pieces to tie together. If you have questions about the code, email me at <dseita@andrew.cmu.edu> or write a GitHub issue report with details about what you want to do, and I will do my best to help you out.

Excitingly, there have been multiple follow-up works that have used our code:

• Our (roughly) concurrent VisuoSpatial Foresight work (RSS 2020).

• A team of researchers from our lab used this for learning switching criteria for Sim2Real fabric manipulation (CASE 2022).

• An external team of researchers from the Harbin Institute of Technology used our code for fabric folding and box wrapping (RA-L 2022).

March 2020 update: I generated an RGBD dataset for the IROS 2020 paper. You can find it here for Tiers 1, 2, and 3, respectively:

• https://drive.google.com/file/d/1M5LB778ZEAF18455w-LsEkTtiDPz2wtr/view?usp=sharing

• https://drive.google.com/file/d/1TfrOSSmG1HazBnDs18ZBipYphhXSJQGO/view?usp=sharing

• https://drive.google.com/file/d/1qupBQAY0KbN-aJYoDZhy4yu-xyXqgZaD/view?usp=sharing

These files will correspond to these file names (dated from February 09 and February 10):

I strongly suggest using the above data set if you are interested in using data from this work.

The older offline demonstrator data (with color alone or depth alone) can be found HERE (warning: 5.2 GB). Run the command to tar it to get:

This is technically is not needed for DAgger, but it's nice to have data to get the learner policy in a good configuration before we do DAgger. The first three are for depth images, and the last three are for color images (as "depthimg" is False in the file name). All are pickle files that store one list, of which each item in the list contains information about a specific trajectory.

The videos below are taken from rendering software that we use to visualize the simulator. We use the rendering software for taking videos of the simulator and debugging, but not for domain randomization. For that, we export our cloth meshes to Blender. For simulated videos, the fabric plane is blue. (In the real setup, the fabric plane is white foam rubber.)

Note: it sometimes looks like the pulls are pulling "past" the corner but that's an artifact of our renderer and camera angles.

Here are videos of the physical setup. I arrange this by tier, listing tiers 1, 2, and 3, respectively, and show the performance of policies that use color (RGB), depth (D), or RGBD image inputs. We show many videos for completeness, but I recommend looking at the videos below showing the RGBD policy on tier 3 starting configurations.

Here are additional (successful) RGBD policy example rollouts on Tier 3 configurations, at 4X speed, with the camera movement partially stabilized compared to earlier videos.

A common failure case / limitation is when the policy slightly misses grabbing the fabric, as shown in the below left video. As a partial way to address this, we measure structural similarity of the image before and after the action. If it shows that the two images are nearly identical, the next action moves closer to the center, which is usually sufficient for our purposes. Notice that this may require several actions, since there's no guarantee the next action will touch fabric. An alternative would be to simply map it to the nearest fabric pixel, though this is also subject to calibration error.

The below two videos show back-to-back actions in the same episode. This is shown with our older setup where we taped some paper in the background. The newer setup, used for experiments, uses a flat piece of paper with a cut piece to allow the foam rubber to be seen.

@inproceedings{seita_fabrics_2020,

author = {Daniel Seita and Aditya Ganapathi and Ryan Hoque and Minho Hwang and Edward Cen and Ajay Kumar Tanwani and Ashwin Balakrishna and Brijen Thananjeyan and Jeffrey Ichnowski and Nawid Jamali and Katsu Yamane and Soshi Iba and John Canny and Ken Goldberg},

title = {{Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor}},

booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},

Year = {2020}

}

This research was performed at the AUTOLAB at UC Berkeley in affiliation with Honda Research Institute USA, the Berkeley AI Research (BAIR) Lab, Berkeley Deep Drive (BDD), the Real-Time Intelligent Secure Execution (RISE) Lab, and the CITRIS “People and Robots” (CPAR) Initiative, and by the Scalable Collaborative Human-Robot Learning (SCHooL) Project, NSF National Robotics Initiative Award 1734633. The authors were supported in part by Siemens, Google, Amazon Robotics, Toyota Research Institute, Autodesk, ABB, Samsung, Knapp, Loccioni, Intel, Comcast, Cisco, Hewlett-Packard, PhotoNeo, NVidia, and Intuitive Surgical. Daniel Seita is supported by a National Physical Science Consortium Fellowship. We thank Jackson Chui, Michael Danielczuk, Shivin Devgon, and Mark Theis.