So Kuroki, Tatsuya Matsushima, Jumpei Arima, Hiroki Furuta,
Yutaka Matsuo, Shixiang Shane Gu, and Yujin Tang
The University of Tokyo, Japan, Matsuo Institute, Japan, Google Research, Brain Team
IEEE Robotics and Automation Letters (will be presented 2023 IROS)
While natural systems often present collective intelligence that allows them to self-organize and adapt to changes, the equivalent is missing in most artificial systems. We explore the possibility of such a system in the context of cooperative 2D push manipulations using mobile robots. Although conventional works demonstrate potential solutions for the problem in restricted settings, they have computational and learning difficulties. More importantly, these systems do not possess the ability to adapt when facing environmental changes. In this work, we show that by distilling a planner derived from a differentiable soft-body physics simulator into an attention-based neural network, our multi-robot push manipulation system achieves better performance than baselines. In addition, our system also generalizes to configurations not seen during training and is able to adapt toward task completions when external turbulence and environmental changes are applied.
github -> https://github.com/Kuroki1931/CIOM
arXiv -> https://arxiv.org/abs/2211.15136
Journal -> https://ieeexplore.ieee.org/document/10080994
youtube -> https://www.youtube.com/watch?v=dU-pbRlgaik&t=25s
Video
new_video.mp4*We recommend you see this video with full screen for the high resolution.
Particle-based Planner
Push box
× 4
Push rope
× 30
Push cloth
× 30
Pipeline
Generalization for the number of robots
3 robots
4 robots
5 robots
Generalization for kidnapping
× 30
No kidnapping
× 30
kidnapping
after 1st closed loop
× 30
kidnapping
after 2nd closed loop
× 30
kidnapping
after 3rd closed loop
The upper right robot start moving to the left direction at the next closed loop right after the center robot is kidnapped.
Attention matrices
Comparing (a.2) and (b.2), the attention weights to the nearest robots suddenly became larger right after kidnapping, proving that the robots are adjusting and relying more on the information from the rest of the robots to accomplish the task.
BibTex
@article{kuroki2023collective,
title={Collective Intelligence for 2D Push Manipulations With Mobile Robots},
author={Kuroki, So and Matsushima, Tatsuya and Arima, Jumpei and Furuta, Hiroki and Matsuo, Yutaka and Gu, Shixiang Shane and Tang, Yujin},
journal={IEEE Robotics and Automation Letters},
year={2023},
publisher={IEEE}
}