LAN-CB

Compositional Coordination for Multi-Robot Teams with Large Language Models

Understand the Paper in 3 Minutes

video.mp4

Abstract

Multi-robot coordination has traditionally relied on a mission-specific and expert-driven pipeline, where natural language mission descriptions are manually translated by domain experts into mathematical formulation, algorithm design, and executable code. This conventional process is labor-intensive, inaccessible to non-experts, and inflexible to changes in mission requirements. Here, we propose LAN2CB (Language to Collective Behavior), a novel framework that leverages large language models (LLMs) to streamline and generalize the multi-robot coordination pipeline. LAN2CB transforms natural language (NL) mission descriptions directly into executable Python code for multi-robot systems through two core modules: (1) Mission Analysis, which parses user instructions into behavior trees, and (2) Code Generation, which uses these behavior trees and a structured knowledge base to generate mission management and robot control code. We further introduce a dataset of natural language mission specifications to support development and benchmarking. Experiments in both simulation and real-world environments demonstrate that LAN2CB enables robust and flexible multi-robot coordination from natural language, significantly reducing manual engineering effort and supporting broad generalization across diverse mission types.

Overview

A comprehensive workflow demonstration

Simulation Results

Mission #1

Mission Description:

The left three robots collaboratively visit regions 0, 1, and 2, while the right three robots gather around region 3. Afterward, all robots enclose region 3.

Behavior Tree:

Mission #2

Mission Description:

All robots herd all purple dynamic objects into region 1. Then, three robots block the left side of region 1, and two robots block the right side.

Behavior Tree:

Mission #3

Mission Description:

The left four robots herd the left two purple dynamic objects into region 0, while the right four robots herd the right two purple dynamic objects into region 0. Then, all robots enclose region 0.

Behavior Tree:

Mission #4

Mission Description:

All robots collaboratively form 'H' first, then form 'I'.

Behavior Tree:

Mission #5

Mission Description:

The left two robots move to their targets in a zigzag pattern, while the right two robots move in a spiral pattern.

Behavior Tree:

Mission #6

Mission Description:

Left, middle, and right pairs of robots each follow their corresponding purple object for 10 seconds, then enter region 0 from the left, top, and right edges, respectively.

Behavior Tree:

Mission #7

Mission Description:

The left three robots visit points near the top-right of region 0, but if any approach within 1.5 units of the forbidden region, they immediately move to region 0. The right three robots enclose region 1 first, and if the left robots fail their task, the right robots take over.

Behavior Tree:

Mission #8

Mission Description:

Left four robots herd the left two purple dynamic objects to region 0, while right four herd the right two to region 1. If the centers of the two groups of dynamic objects are within 3 units, all robots herd all objects to region 2.

Behavior Tree:

Mission #9

Mission Description:

All robots herd all dynamic objects to cover 60% area of region 0.

Behavior Tree:

Real World Demonstrations

RM1.mp4

Real-World Mission #1

Mission Description:

The robot moves to the target point in a zigzag pattern.

Behavior Tree:

RM2.mp4

Real-World Mission #2

Mission Description:

Robot 1 follows Robot 2 for 20 seconds, and then enters the orange region via its right edge.

Behavior Tree:

RM4.mp4

Real-World Mission #3

Mission Description:

Three robots herd the target robot into the orange region. Once the target robot is inside, the three robots must enclose the orange region.

Behavior Tree:

P2_RM4.mp4

Real-World Mission #4

Mission Description:

All robots collaboratively visit the top-left and bottom-right corners of right two regions. Then, all robots form a reverse 'C' around the left region.

Behavior Tree:

Page updated

Google Sites

Report abuse