International Conference of Machine learning (ICML) 2023
The complexity of human dexterity has attracted attention from multiple fields. Still, much is to be understood about how hand manipulation behaviors emerge. Studying manipulation behaviors with a physiological realistic hand model (MyoHand) at scale, MyoDex is an attempt at recovering generalizable priors for dexterous manipulation.
In contrast to prior works demonstrating isolated postural and force control, here we demonstrate musculoskeletal agents (MyoDex) exhibiting contact-rich dynamic dexterous manipulation behaviors in simulation (MyoSuite).
Unlike commonly used formulation of behavior synthesis which requires task-specific engineering (rewards/ curriculum/ task tuning), MyoDex uses only exemplar object trajectories and associated Pre-Grasps to learn task policies. The resulting hyper-parameter-free formulation that requires no task-tuning allows us to scale dexterous manipulation to a large repertoire of behaviors. We also demonstrate that a student-teacher framework while successful at a smaller scale, fails at recovering priors that generalize more broadly.
Aligned with human development, simultaneous learning of multiple tasks imparts physiological coordinated muscle contractions i.e., muscle synergies. Indeed, MyoDex has multiple synergies shared between tasks while the expert and student do not.
Muscle synergies, are not only shared amongst in-domain tasks but are also effective in out-of-domain tasks. Indeed, in zero-shot settings, MyoDex -- without having any information on the object -- is able to initiate manipulations.
⮕ Figure: Showcase of MyoDex representations generalizing zero shot to novel objects. It exhibits basic interaction behaviors involving picking and investigating the object. These representations can be quickly fine-tuned to solve previously unsolved tasks.
MyoDex-ICML-ZeroShot.mp4Examples of zero-shot generalization to new out-of-domain tasks
By leveraging MyoDex prior, fine-tuning allow to solve 37 new tasks, 21 of which are unable to be solved by experts. Not only MyoDex allows learning more tasks, but it does it 4x faster than baselines.
All_FT_Collage.mp4
To further investigate the applicability of our approach to other high dimensional systems, we set out to build a generalizable representation for AdroitHand (Rajeshwaran, Kumar et. al. 2018) commonly studied in robot learning. Adroit is a 24 degree of freedom (DoF) modified shadow hand with 4 extra DoF at the distal joint. Following the approach behind MyoDex, a general representation of manipulation prior - AdroitDex - was obtained. We use the same 14 tasks that we used for training MyoDex. In the figure below we show the performance of AdroitDex on 34 unseen tasks on the TCDM benchmark (Dasari et al 2022). AdroitDex beats previously reported SOTA on TCDM benchmarks (Dasari et al 2022) while being 5x more sample efficient!
Figure: [Left] mean and std performance across all 34 tasks, [Center] Learned behavior showcasing a natural grasp, [Right] Learned behavior showcasing the emegent in-hand finger maneuvers involved in solving the task.
Synergies (weighted co-activations of muscles) were extracted at 12.5k iterations of the jointly trained multi-tasks.
⮕ Figure on the right show 12 synergies played with a stepwise activation. This animation show how fingers and wrist are affected when muscles are activated as a unit.
