In-Mouth Robotic Bite Transfer
With Visual and Haptic Sensing
Embedded Files
Abstract
Abstract
Assistance during eating is essential for those with severe mobility issues or eating risks. However, dependence on traditional human caregivers is linked to malnutrition, weight loss, and low self-esteem. For those who require eating assistance, a semi-autonomous robotic platform can provide independence and a healthier lifestyle. We demonstrate an essential capability of this platform: safe, comfortable, and effective transfer of a bite-sized food item from a utensil directly to the inside of a person’s mouth. Our system uses a force-reactive controller to safely accommodate the user’s motions throughout the transfer, allowing full reactivity until bite detection then reducing reactivity in the direction of exit. Additionally, we introduce a novel dexterous wrist-like end effector capable of small, unimposing movements to reduce user discomfort. We conduct a user study with 11 participants covering 8 diverse food categories to evaluate our system end-to-end, and we find that users strongly prefer our method to a wide range of baselines.
Supplementary Video
Supplementary Video
Wrist Mount
Wrist Mount
Our custom wrist mount is designed to work with a Franka Emika Panda robot end effector. Here's what you need to build your own:
Human Oracle involves the user being fed by another human, which serves as a reasonable upper bound for our robotic feeding system.
tony_pineapple_human_1.MOVpineapple chunk
tony_tomato_human_1.MOVcherry tomato
20_broccoli_human_trial_1.MOVbroccoli piece
20_tofu_human_trial_1.MOVtofu chunk
Fixed Pose is not an in-mouth bite-transfer method from prior work. Instead, the robot moves the fork to a position outside the user’s mouth and allows the user to take a bite on their own.
20_cheesecake_fixed_trial_1.MOVcheesecake
20_broccli_fixed_trial_2.MOVbroccoli piece
5_tomato_fixed_trial_2.MOVcherry tomato
5_blueberry_fixed_trial_2.MOVblueberry
Our Method involves a novel wrist-like end-effector, capable of small, unimposing movements to reduce the user discomfort and a phased force-reactive controller to safely accommodate users' motions. It is substantially preferred over baselines across 11 participants and 8 different bite-sized food items in the user study.
carrot_our.movcarrot
pineapple_our_1.movpineapple chunk
5_tofu_our_trial_2.MOVtofu chunk
strawberry_our_1.movstrawberry
Less Reactive is an ablation of our system, which uses less force reactivity throughout the whole feeding process.
blueberry_less_reactive_1.movblueberry
strawberry_less_reactive_1.movstrawberry
pineapple_less_reactive_1.movpineapple chunk
20_cheesecake_less_trial_1.MOVcheesecake
More Reactive is an ablation of our system, which uses more force reactivity throughout the feeding process and is representative to the in-mouth bite transfer work[1].
20_broccoli_more_trial_1.MOVbroccoli piece
carrot_more_reactive_1.movcarrot
strawberry_more_fail_to_bite.movstrawberry
tofu_more_reactive_1.movtofu chunk
Failure Modes
strawberry_more_fail_to_bite_2.movBite Failure
Bite Failure
In Bite Failure, users fail to take a bite of the food. This can occur when either the user retracts early due to mistrust or fails to take a bite before the fork exits.
carrot_less_inaccurate_location.movImprecise
Imprecise
Imprecise denotes the face detector localizing the mouth inaccurately, causing a near-miss or off-center transfer.
carrot_fall_our_short.movDrop
Drop
In Drop, the food falls off the fork during the transfer process.
Page updated
Google Sites
Report abuse