Reactive Human-to-Robot Handovers of Arbitrary Objects
Wei Yang, Chris Paxton, Arsalan Mousavian, Yu-Wei Chao, Maya Cakmak, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA) 2021, Xi’an, China
2021 IEEE ICRA Best Paper Award on Human-Robot Interaction (HRI)
ICRA21' Presentation
arXiv
NVIDIA Blog
Human-robot object handovers have been an actively studied area of robotics over the past decade. However, very few techniques and systems have addressed the challenge of handing over diverse objects with arbitrary appearance, size, shape, and rigidity.
We present a vision-based system that enables reactive human-to-robot handovers of unknown objects. Our approach combines closed-loop motion planning with real-time, temporally-consistent grasp generation to ensure reactivity and motion smoothness. Our system is robust to different object positions and orientations, and can grasp both rigid and non-rigid objects.
3 Minute Overview
User Study
We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects, a user study with naive users (N=6) handing over a subset of 15 objects.
Experiment 1: Handovers of A Fixed Set of 10 Household Objects (Speed: 4X)
Round 1 involves a fixed set of 10 objects we selected from our household objects benchmark. Objects were handed in a predefined order one by one. The users were told that the robot will start to move towards the object to take it from them once the object is still.
Object list: newspaper, medicine box, mug, plate, toothpaste, pen, spoon, remote, scissors, hand towel
Experiment 2: Handovers of Random Objects Selected by Participants (Speed: 4X)
Round 2 involved open-ended handovers where users were asked to pick any five objects they wanted from the rest of the household objects and hand them over one-by-one to the robot.
plastic container, toothbrush, watch, magazine, t-shirt
bowl, small pillow, pump bottle, watch, credit card
letter, small pillow, watch, disposable bottle, cellphone
t-shirt, small pillow, cellphone, toothbrush, watch
magazine, small pillow, pump bottle, watch, plastic container
disposable bottle, magazine, plastic container, pump bottle, small pillow
Experiment 3: Other Objects in the Lab or Personal Belongings (Speed: 4X)
Optionally, participants were encouraged to handover objects they could find in the lab or their personal belongings to the robot. The following videos show handovers of objects outside of our benchmarks chosen by our participants.
power drill, banana
disinfectant wipes, plastic box, screwdriver, tape (white), diamond block
stapler, tape (red), pliers, mouse, sticky notes, diamond block
bottled water, pitcher, wire reel, cube, screwdriver