Abstract
Efficient mission planning for cooperative systems involving Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) requires addressing energy constraints, scalability, and coordination challenges between agents. UAVs excel in rapidly covering large areas but are constrained by limited battery life, while UGVs, with their extended operational range and capability to serve as mobile recharging stations, are hindered by slower speeds. This heterogeneity makes coordination between UAVs and UGVs critical for achieving optimal mission outcomes. In this work, we propose a scalable deep reinforcement learning (DRL) framework to address the energy-constrained cooperative routing problem for multi-agent UAV-UGV teams, aiming to visit a set of task points in minimal time with UAVs relying on UGVs for recharging during the mission. The framework incorporates sortie-wise agent switching to efficiently manage multiple agents, by allocating task points and coordinating actions. Using an encoder-decoder transformer architecture, it optimizes routes and recharging rendezvous for the UAV-UGV team in the task scenario. Extensive computational experiments demonstrate the framework's superior performance over heuristic methods and a DRL baseline, delivering significant improvements in solution quality and runtime efficiency across diverse scenarios. Generalization studies validate its robustness, while dynamic scenario highlights its adaptability to real-time changes with a case study. This work advances UAV-UGV cooperative routing by providing a scalable, efficient, and robust solution for multi-agent mission planning.
Problem Statement
Illustration of the fuel-constrained UAV-UGV cooperative routing problem. UAV-specific task points (red hollow circles) and ground task points (blue solid circles) must be visited, with rendezvous points (black circles) for UAVs recharging on the UGVs. The goal is to plan UAV and UGV routes (red dashed and blue solid paths, respectively) to minimize total mission time while adhering to UAV fuel and UGV speed constraints.
Deep Reinforcement Learning Framework
Architecture of the proposed transformer network. The encoder consists of three attention layers that process raw input data to generate node embeddings. The decoder employs an agent selection strategy to determine the active agent type (UAV or UGV) and constructs a context vector based on the current state of the selected agent. It uses the input embeddings and the context vector to determine action for the selected active agent.
Agent selection strategy during the mission progress
Experiments