Decluttering and Sorting Lego Blocks with the Toyota Human Support Robot
Grant Wang, Zisu Dong, Chris Powers, Michael Laskey, Ajay Tanwani*, Ken Goldberg
AUTOLAB, UC Berkeley. *contact: ajay.tanwani@berkeley.edu
Welcome to our site that highlights our work on identifying lego blocks in cluttered environments and placing those objects in desired locations with the HSR (Human Support Robot). Below you will find (1) the problem statement (2) our approach to this decluttering problem (3) a video that demonstrates preliminary results of a successful run of our approach on legos (4) Our results from the TRI (Toyota Robotics Institute) decluttering competition (we placed 2nd !)
Problem
Problem Statement: Identify objects in cluttered environments and place them in their desired location (e.g. sort tool and place in toolbox, sort cluttered legos and put them in their respective bins as seen below)
Our Approach
Below you will find a simple to visualize pipeline flow of our approach. This pipeline is still subject to change and improvement as we run more grasps and tests on the Toyota HSR, but for the time being this is the approach utilized in video of the preliminary experimental results section of this site. All tests are run on the Toyota HSR.
Experimental Results
Below is a run of the Toyota HSR decluttering a pile of 7 lego duplo bricks and then sorting them based on color in 4 respective bins (red/orange, blue/cyan, green/green yellow, and yellow bins). The robot singulates the pile 3 times through the run to obtain better grasp points and successfully sorts 6 of the 7 different colored lego duplo bricks in their correct bins. At each iteration, the robot takes time to collect an image of the environment, compute a grasp point, determine whether to singulate, and then either execute a grasp or singulate the pile of objects. The total run-time was 15m 21s, while the video is sped up 8x. Failure cases include (1) the robot misidentifying 1 yellow lego duplo as orange (due to poor lighting in the environment) and (2) 1 red lego duplo falling out of the bin due to collision with another lego duplo dropped in the same bin. These are failure cases we will tackle as the project progresses.
Competition Results
We competed in the Toyota Robotics Institute decluttering competition in August 2018 and took 2nd place. Other teams competing included Stanford, MIT, UMichigan, UT Austin. Here are our results, where we decluttered 3 lego blocks, and successfully placed all 3 in their corresponding colored cubby bin.
TRI Challenge 1.mp4