CoViS-Net
A Cooperative Visual Spatial Foundation Model for Multi-Robot Applications
CoViS-Net is a decentralized, real-time, multi-robot visual spatial model that learns spatial priors from data to provide relative pose estimates and bird's-eye-view representations. We demonstrate its effectiveness in real-world multi-robot formation control tasks.
![](https://www.google.com/images/icons/product/drive-32.png)
Summary
We show the functioning of our model on a multi-agent pose control task. In our in-door laboratory, we render predicted poses and uncertainty estimates on top of one leader and two follower robots.
Indoor Experiments
We run our model on a variety of indoor scenes, with a remote-controlled leaders and up to four followers configured to stay at a fixed relative pose.
![](https://www.google.com/images/icons/product/drive-32.png)
![](https://www.google.com/images/icons/product/drive-32.png)
Outdoor Experiments
While our model was trained exclusively on indoor data, we show that it generalizes to outdoor experiments.
Heterogeneous Multi-Robot System
We show the flexibility of our model by deploying it in a heterogeneous multi-robot experiment. Our platform-agnostic model generalizes to multiple different robot platforms.
![](https://www.google.com/images/icons/product/drive-32.png)
We show quantitative evaluations of the BEV representation prediction on scenes of our real-world dataset. The top row shows the image for each node, the middle row the ground-truth poses in the coordinate frame of each node, and the bottom row pose predictions and pose uncertainty with the BEV representation prediction in the background.
![](https://www.google.com/images/icons/product/drive-32.png)
Corridor A
![](https://www.google.com/images/icons/product/drive-32.png)
Corridor B
![](https://www.google.com/images/icons/product/drive-32.png)
Office A
![](https://www.google.com/images/icons/product/drive-32.png)
Office B
![](https://www.google.com/images/icons/product/drive-32.png)
Outdoor
![](https://www.google.com/images/icons/product/drive-32.png)
Study A
![](https://www.google.com/images/icons/product/drive-32.png)
Study B
![](https://www.google.com/images/icons/product/drive-32.png)
Sunny