Reactive Human-to-Robot Handovers of Arbitrary Objects

Wei Yang, Chris Paxton, Arsalan Mousavian, Yu-Wei Chao, Maya Cakmak, Dieter Fox

IEEE International Conference on Robotics and Automation (ICRA) 2021, Xi’an, China

2021 IEEE ICRA Best Paper Award on Human-Robot Interaction (HRI)

ICRA21' Presentation



Human-robot object handovers have been an actively studied area of robotics over the past decade. However, very few techniques and systems have addressed the challenge of handing over diverse objects with arbitrary appearance, size, shape, and rigidity.

We present a vision-based system that enables reactive human-to-robot handovers of unknown objects. Our approach combines closed-loop motion planning with real-time, temporally-consistent grasp generation to ensure reactivity and motion smoothness. Our system is robust to different object positions and orientations, and can grasp both rigid and non-rigid objects.

3 Minute Overview

User Study

We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects, a user study with naive users (N=6) handing over a subset of 15 objects.

Experiment 1: Handovers of A Fixed Set of 10 Household Objects (Speed: 4X)

Round 1 involves a fixed set of 10 objects we selected from our household objects benchmark. Objects were handed in a predefined order one by one. The users were told that the robot will start to move towards the object to take it from them once the object is still.

Object list: newspaper, medicine box, mug, plate, toothpaste, pen, spoon, remote, scissors, hand towel

Experiment 2: Handovers of Random Objects Selected by Participants (Speed: 4X)

Round 2 involved open-ended handovers where users were asked to pick any five objects they wanted from the rest of the household objects and hand them over one-by-one to the robot.

plastic container, toothbrush, watch, magazine, t-shirt

bowl, small pillow, pump bottle, watch, credit card

letter, small pillow, watch, disposable bottle, cellphone

t-shirt, small pillow, cellphone, toothbrush, watch

magazine, small pillow, pump bottle, watch, plastic container

disposable bottle, magazine, plastic container, pump bottle, small pillow

Experiment 3: Other Objects in the Lab or Personal Belongings (Speed: 4X)

Optionally, participants were encouraged to handover objects they could find in the lab or their personal belongings to the robot. The following videos show handovers of objects outside of our benchmarks chosen by our participants.

power drill, banana

disinfectant wipes, plastic box, screwdriver, tape (white), diamond block

stapler, tape (red), pliers, mouse, sticky notes, diamond block

bottled water, pitcher, wire reel, cube, screwdriver