Multi-UAV Trajectory Generation using MALRL 

In the future, multiple UAVs (Unmanned Aerial Vehicles) are expected to operate concurrently within the same airspace, either autonomously or as part of independent fleets, performing various tasks. The imperative of avoiding collisions between UAVs and manned aircraft will present new challenges for industries, academia, and regulatory bodies.

Researchers at Sapienza University of Rome, under the coordination of Prof. Luca Iocchi, developed a machine learning algorithm for conflict mitigation as part of the BUBBLES project.

The algorithm comprises four main phases, with the first involving the generation of trajectory data pertinent to the operational scenario.

MALRL (Multiple Abstraction Layers Reinforcement Learning) enables the generation and storage of realistic trajectories, which can be utilized as training or testing data. MALRL is based on multiple abstraction levels, aiming at exploiting existing Reinforcement Learning 2D environments (first layer) to integrate the results with a 3D simulator (second and third layer).

The studied operational scenario is based on Barcelona city but MALRL can be extended to any area that needs to be covered with grid pattern flights. 

Horizontal and vertical separation has been enforced while generating trajectories, using Q-Learning for the former and a deterministic policy for the latter. When each UAV has an associated altitude and 2D path, AirSim is used to let a UAV simulate the flight on the path and obtain as output 3D realistic trajectories. The final abstraction layer incorporates OpenStreetMap, Blender, and AirSimGeo to add GPS referencing for UAV waypoints.