FUSION: Safety-aware Causal Representation for Trustworthy Reinforcement Learning in Autonomous Driving

The Learning from Demonstration (LfD) paradigm has exhibited notable efficacy in addressing sequential decision-making problems within the domain of autonomous driving. However, consistently achieving safety in varying traffic contexts, especially in safety-critical scenarios, poses a significant challenge due to the long-tailed and unforeseen scenarios absent from offline datasets. In this paper, we introduce the saFety-aware strUctured Scenario representatION (FUSION), a pioneering methodology conceived to facilitate the learning of an adaptive end-to-end driving policy by leveraging structured scenario information. FUSION capitalizes on the causal relationships between decomposed reward, cost, state, and action space, constructing a framework for structured sequential reasoning under dynamic traffic environments. Our rigorous evaluations, conducted in both in-distribution and generalization settings, demonstrate the efficacy of FUSION compared to contemporary state-of-the-art baselines. Empirical evidence attests that FUSION significantly enhances the autonomous driving agent's decision-making capability, even in the face of challenging and unseen driving scenarios. Furthermore, our ablation studies reveal noticeable improvements in the integration of causal representation into the safe offline RL process.

See below for more matrix visualization results