Abstract

End-to-end autonomous driving (AD) systems integrate complex decision-making processes. Assessing the safety of these systems against potential security threats, including backdoor attacks, is a stepping stone for real-world deployment. However, traditional methods focus on static triggers, which do not adequately reflect the dynamic nature of these systems and could be impractical to deploy in the real world. To address these limitations, we propose a novel backdoor attack against the end-to-end AD systems that leverage multi-vehicles' trajectories as triggers. We employ different behavior models and their configurations to generate the trigger trajectories, which are then quantitatively evaluated using temporal logic specifications. This evaluation guides the subsequent perturbations to the behavior model configurations. Through an iterative process of regeneration and re-evaluation, we can refine and generate realistic and plausible trigger trajectories that involve multiple vehicles' complex interactions.

Furthermore, we develop a negative training strategy by incorporating patch trajectories that share similarities with the triggers but are designated not to activate the backdoor. We thus enhance the stealthiness of the attack, refining the system’s responses to trigger scenarios. Through extensive empirical studies using offline reinforcement learning (RL) driving agents with various trigger patterns and target action designs, we demonstrate the flexibility and effectiveness of our proposed attack, showing the under-exploration of existing end-to-end AD systems' vulnerabilities to such multi-vehicle-based backdoor attacks.

Videos

In the MetaDrive simulator, we showcase two demonstrations—one from a top-down perspective and one in 3D mode—to illustrate how coordinated multi-vehicle trajectories can trigger backdoor attacks in offline RL agents. Specifically, the triggers are:

1. Two cars synchronously bypass the ego car

2. One car suddenly brakes while another overtakes

Once these triggers occur, the ego car executes one of the following target actions:

3. A sudden left turn

4. An abrupt brake

Top-down view

Trigger: two cars synchronously bypass the ego car 

Target action: the green ego car suddenly turns left

 Trigger: one car suddenly brakes, the other overtakes

Target action: the green ego car suddenly turns left

Trigger: two cars synchronously bypass the ego car

Target action: the green ego car suddenly brakes

Closed-loop 3D mode 

Trigger: two cars synchronously bypass the ego car 

Target action: the red ego car (in the middle) suddenly turns left