By Daniel Seita et al. If you have questions on how use the code or the project in general, email me at seita@berkeley.edu.

Updates:

• 05/31/2019: The paper has been submitted to ISRR 2019.
• 09/16/2019: The paper has been accepted and we have submitted the final version.

Here, you can find the paper, data, and videos.

• The paper which was submitted: https://drive.google.com/file/d/1UmMdNHcEsDNmKJoVUCQGKjxHAU1ngCwl/view?usp=sharing
• The arXiv is here, which has the latest version (it was originally under an old title): https://arxiv.org/abs/1809.09810

This research was conducted at the AUTOLAB at UC Berkeley. http://autolab.berkeley.edu/

BibTex:

@inproceedings{seita_bedmake_2019,

    author = {Daniel Seita and Nawid Jamali and Michael Laskey and Ajay Kumar Tanwani and Ron Berenstein and Prakash Baskaran and Soshi Iba and John Canny and Ken Goldberg},

    title = {{Deep Transfer Learning of Pick Points on Fabric for Robot Bed-Making}},

    booktitle = {International Symposium on Robotics Research (ISRR)},

    Year = {2019}

}

Code:

• Code for bed-making: https://github.com/DanielTakeshi/IL_ROS_HSR
• Code for neural network training: https://github.com/DanielTakeshi/fast_grasp_detect
• Both of the repositories above refer to an arXiv version of the paper, which contains the most up to date version.

Data:

• Here is the data you can use for training the grasp network: https://drive.google.com/open?id=1l5Aup1AnMNPzlcRVYZu9GYjBOsD9D-z2
• And here is the data you can use for training the success network: https://drive.google.com/open?id=1D2OUHKOY26TetsBodmtMhAxCKT65ZDU6 [Note: we used to emphasize a 'success / transition network' for this research, but we de-emphasized it in favor of a better understanding of just the grasp (or pick point) network. However, we leave the data here for completeness. The transition network was getting 99%-ish accuracy so it was nearly perfect, but we could have used alternative methods (e.g., whether a marker is covered or not) which is why we de-emphasized the discussion in subsequent research.]

If you unzip the data, you'll get stuff that looks like this for the grasp network and then the success network:

These are split into 10 pickle files for each network. Each of the 10 files (per neural network) is a standard pickle file (requires python 2.7 to load, not python 3, sorry, we were forced to use this due to code dependencies) which represents a standard python list. The lists have length equal to whatever is listed in the pickle file name. The data has already been shuffled and split into cross validation folds, which is what the `cv` represents in the file names. Of course, during training, the data needs to be further shuffled for minibatches. Each data point in the lists is a dictionary with `c_img` and `d_img` keys representing RGB and depth images. The latter is what we want to use. These also have labels within the dictionary.

Coverage Results:

• Here's the data used for evaluating coverage: https://drive.google.com/drive/folders/1JsrMUh8fFog4LZurppZvS28THMbh3vQr?usp=sharing
• It is split into several directories, with hopefully self-explanatory names. The coverage is listed in the file names for the `.png` images.

VIDEO SUBMISSION

The video below is the main video we submitted.

OTHER VIDEOS

This shows a clip of a successful rollout with the teal blanket. Taken with an iPhone, so it's a bit shaky.

This shows one of the earlier failure cases with the offsets not being set correctly. This led us to adjust the offset of the gripper height as needed. In the future, we will also use a soft pad instead of a hard bed frame top.

Here's why the analytic baseline often gets good coverage. It doesn't grasp a corner but grasps a point such that the pull results in high coverage.