Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator
ACM/IEEE International Conference on Human Robot Interaction (HRI) 2024
Best Paper Award in Systems
[arXiv][Video] [Build Instructions and Code]
This paper builds on work previously presented at the International Conference on Robotics and Automation 2023. More information is located on the HAT1 Website.
Project Video
Abstract
Teleoperation of mobile manipulators within a home environment can significantly enhance the independence of individuals with severe motor impairments, allowing them to regain the ability to perform self-care and household tasks. There is a critical need for novel teleoperation interfaces to offer effective alternatives for individuals with impairments who may encounter challenges in using existing interfaces due to physical limitations. In this work, we iterate on one such interface, HAT (Head-Worn Assistive Teleoperation), an inertial-based wearable integrated into any head-worn garment. We evaluate HAT through a 7-day in-home study with Henry Evans, a non-speaking individual with quadriplegia who has participated extensively in assistive robotics studies. We additionally evaluate HAT with a proposed shared control method for mobile manipulators termed Driver Assistance and demonstrate how the interface generalizes to other physical devices and contexts. Our results show that HAT is a strong teleoperation interface across key metrics including efficiency, errors, learning curve, and workload.
Henry, who is a non-speaking individual with quadriplegia, uses HAT, a head-worn assistive interface for mobile manipulators. The first image provides a system overview. The interface records and sends Henry's head orientation angles over Bluetooth to a companion laptop, which computes and transmits actuator velocities to the robot over WiFi. A clicker, plugged into the companion laptop, is used by Henry to switch between modes, distinct operational states of the HAT system. The second and third images show snippets from a task conducted during the study. In the second image, Henry uses the interface to move a blanket below his knees. In the third image, Henry uses HAT to grasp a tissue to wipe his face and throw the tissue in a trash can
Left: Henry wearing the chin strap with the HAT interface embedded. Right: The HAT Bluetooth communication dongle and foam insert with all electronics labelled.
Human Study Task Videos
Blanket+Tissue+Trash
blankettissuetrash_day1_edited_5.5x.mp4Day 1
Total task time: 16 mins and 30 seconds
blankettissuetrash_day7_edited_5.5x.mp4Day 7
Total task time: 8 mins and 44 seconds
For the blanket+tissue+trash task, which was conducted every day of the study, there is a noticeable downward trend in task completion times over the week. From day 1 to day 7, there was a 47% (7m 46s) decrease in task time.
Scratching Itch
scratching_itch_day1_edited_4x.mp4Day 1
Total task time: 8 mins and 49 seconds
scratching_itch_edited_day7_4x.mp4Day 7
Total task time: 5 mins and 17 seconds
For the scratching itch task, which was conducted on the first day and repeated on the seventh day of the study, there is a clear improvement with a 40% (3m 32s) reduction in task completion time.
Feeding
feeding_edited_day2.mp4Day 2
Total task time: 9 mins and 8 seconds
feeding_day7_edited_4x.mp4Day 7
Total task time: 7 mins and 18 seconds
For the feeding task, which was conducted on the first day and repeated on the seventh day of the study, there is a clear improvement with a 20% (1m 50s) reduction in task completion time.
Removing Soiled Towel
removingsoiledtowel_day2_edited_3x.mp4Day 2
Total task time: 8 mins and 1 second
removingsoiledtowel_day7_edited_3x.mp4Day 7
Total task time: 4 mins and 26 seconds
For the removing soiled towel task, which was conducted on the first day and repeated on the seventh day of the study, there is a clear improvement with a 45% (3m 35s) reduction in task completion time.
Driver Assistance: Fetching Red Bull
redbull_edited_da_4x.mp4Driver Assistance
Total task time: 3 mins and 18 seconds
redbull_teleop_edited_4x.mp4Full Teleoperation
Total task time: 11 mins and 3 seconds
For the fetching Red Bull task, Driver Assistance, our proposed shared control method, reduced task completion time by 70% (7m 45s) and was rated much lower (better) on the NASA TLX scale. Additionally, our participant reported still feeling in control. The median response for the Likert Item "I felt like I was in control while using driver assistance" was a 5.
Open Tasks
wipingtable_day2_edited_2x.mp4Grasping Towel+Wiping Table
Total task time: 5 mins and 46 seconds
openingfridge_edited_4x.mp4Opening Fridge
Total task time: 9 mins and 18 seconds
Cursor Control: Searching for a Video
cursorcontrol_day3_edited_realtime.mp4We evaluated the generalizability of HAT to other physical devices such as a computer cursor. Our participant noted that it was "pretty close" in comparison to his current head tracker and advantageous in not requiring frequent calibration.