TempFuser: Learning Agile, Tactical, and Acrobatic Flight Maneuvers Using a Long Short-Term Temporal Fusion Transformer

1Korea Advanced Institute of Science and Technology

{hynkis,hcshim}@kaist.ac.kr

Arxiv paper

Abstract

Dogfighting is a challenging scenario in aerial applications that requires a comprehensive understanding of both strategic maneuvers and the aerodynamics of agile aircraft. The aerial agent needs to not only understand tactically evolving maneuvers of fighter jets from a long-term perspective but also react to rapidly changing aerodynamics of aircraft from a short-term viewpoint. In this paper, we introduce TempFuser, a novel long short-term temporal fusion transformer architecture that can learn agile, tactical, and acrobatic flight maneuvers in complex dogfight problems. Our approach integrates two distinct temporal transition embeddings into a transformer-based network to comprehensively capture both the long-term tactics and short-term agility of aerial agents. By incorporating these perspectives, our policy network generates end-to-end flight commands that secure dominant positions over the long term and effectively outmaneuver agile opponents. After training in a high-fidelity flight simulator, our model successfully learns to execute strategic maneuvers, outperforming baseline policy models against various types of opponent aircraft. Notably, our model exhibits human-like acrobatic maneuvers even when facing adversaries with superior specifications, all without relying on explicit prior knowledge. Moreover, it demonstrates robust pursuit performance in challenging supersonic and low-altitude situations.

Learning Tactical and Agile Flight Maneuvers in Aerial Dogfights

Air-to-air combat is the tactical art of maneuvering a fighter agent to reach a position to aim at an opponent. It is also known as dogfighting, as in most cases, each fighter jet pursues the tail of the other in short-range combat situations.

For successful dogfights, the agent requires a combination of situational awareness, strategic planning, and maneuverability from long and short-term perspectives.

Firstly, the agent has to plan its tactical position by understanding the opponent's long-term trajectories. Naive chasing after the adversary's immediate positions may provide a temporary advantage, but it can eventually leave itself in a vulnerable position later. Therefore, the agent should constantly assess the opponent's long-term maneuvers, react to their actions, and strategically position itself to gain an advantage over the adversary.

Secondly, the agent needs to have the ability to comprehend the agile maneuverability of the aircraft from a short-term dynamics perspective. Modern fighter jets are engineered to possess high maneuverability, enabling them to swiftly alter direction and speed, resulting in rapid changes in the engagement situation. Therefore, in order to maintain an advantageous position against the opponent, the agent should promptly grasp both the opponent's agile movements and the agent's own potential maneuvers from a dynamic perspective.

Long Short-Term Temporal Fusion Transformer (TempFuser)

The Long Short-Term Temporal Fusion Transformer, or TempFuser, is a network architecture designed for policy models in aerial dogfights. This architecture uses LSTM-based input embeddings and a Transformer encoder. It processes two types of state trajectories: the long-term temporal trajectory, which represents maneuver-level state transition, and the short-term temporal trajectory, which signifies dynamics-level state transition. Each of these trajectories is embedded using LSTM-based pipelines and then integrated through the Transformer encoder. Subsequently, the encoder output is converted into flight commands using a Multilayer Perceptron (MLP) block and a Gaussian policy architecture.

High-Fidelity Environment using the Digital Combat Simulator (DCS)

We tackle the aerial dogfight problem with Deep Reinforcement Learning (DRL) in the Digital Combat Simulator (DCS), considered one of the most authentic and realistic simulation environments for fighter aircraft. DCS offers a unique platform for configuring a wide range of high-quality aircraft and airborne scenarios. We formulate the dogfighting problem as a reinforcement learning framework and design a reward function that can learn strategic dogfight maneuvers.

We extensively train and validate our network with various opponent aircraft, such as F-15E, F-16, F/A-18A, and Su-27. As a result, we demonstrate that TempFuser learns challenging flight maneuvers in an end-to-end manner and outperforms various opponent aircraft, including those with superior specifications. Additionally, it exhibits robust pursuit performance at low altitudes and high-speed flight scenarios above Mach 1.