Diffusion-EDFs:
Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation

CVPR 2024 (Highlight)

Abstract

Diffusion generative modeling has become a promising approach for learning robotic manipulation tasks from stochastic human demonstrations. In this paper, we present Diffusion-EDFs, a novel SE(3)-equivariant diffusion-based approach for visual robotic manipulation tasks. We show that our proposed method achieves remarkable data efficiency, requiring only 5 to 10 human demonstrations for effective end-to-end training in less than an hour. Furthermore, our benchmark experiments demonstrate that our approach has superior generalizability and robustness compared to state-of-the-art methods. Lastly, we validate our methods with real hardware experiments.

Gallery

Technical Demonstration @ Yonsei University M.E. Grad. Students Conference

(Best Technical Demonstration Award)

Real robot experiments

mug.mp4
bowl.mp4
bottle.mp4

Mug-on-a-hanger

mug_real_denoising (pick).mp4
mug_real_denoising (place).mp4
mug_real_denoising_unseen (pick).mp4
mug_real_denoising_unseen (place).mp4

Bottles-on-a-shelf

bottle_real_denoising_unseen1 (pick).mp4
bottle_real_denoising_unseen1 (place).mp4
bottle_real_denoising_unseen2 (pick).mp4
bottle_real_denoising_unseen2 (place).mp4

Bowls-on-dishes

bowl_real_denoising_red (pick).mp4
bowl_real_denoising_red (place).mp4
bowl_real_denoising_green (pick).mp4
bowl_real_denoising_green (place).mp4
bowl_real_denoising_blue (pick).mp4
bowl_real_denoising_blue (place).mp4

Citation

Please consider citing our paper if you find it helpful. 

@article{ryu2023diffusion,
  title={Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation},
  author={Ryu, Hyunwoo and Kim, Jiwoo and Chang, Junwoo and Ahn, Hyun Seok and Seo, Joohwan and Kim, Taehan and Choi, Jongeun and Horowitz, Roberto},
  journal={arXiv preprint arXiv:2309.02685},
  year={2023}
}