Reinforcement learning (RL) from pixels has achieved strong performance in simulated environments but faces challenges before it can be deployed in real-world, safety-critical domains due to diverse visual conditions and safety constraints. Existing vision-based RL methods often encode task-irrelevant features from observations, which can distract the agent, reduce performance, and limit policy generalization. While approaches have been proposed to tackle this problem, they are not directly suited for policy learning under safety constraints. We propose Safety-Aware Invariant Representations (SAIR), a bisimulation-based approach that learns behaviorally equivalent latent representations while incorporating safety considerations in a Constrained Markov Decision Process (CMDP) setting. To evaluate SAIR, we introduce the Distracting Safety Gymnasium, an openly available safety benchmark with visual distractions based on previous distraction and safety benchmarks. Experimental results show that SAIR outperforms existing SafeRL methods in dynamic distraction settings.