FSA-Net: A Face Swapping Attention Network with Occlusion-Aware Normalization

The main challenges in face swapping are how to preserve and adaptively superimpose attributes of two images. In this work, FSA-Net is proposed to generate photorealistic face swapping. Unlike previous existing face swapping methods, they either ignore the blending attributes, or mismatch the facial keypoint (cheek, mouth, eye, nose, etc.), causes artifacts, which makes the generated face silhouette non-realistic. To address the problem, a novel reinforced multi-aware attention framework, named RMAA framework is proposed, for handling facial fusion and expression occlusion flaws. The framework includes two stages. In the first stage, a new attribute encoder is proposed to extract multiple levels of target face attributes and integrate identities and attributes when synthesizing swapped faces. In the second stage, a new Stochastic Error Refinement (SRE) module is designed to solve the problem of facial occlusion, which is used to repair occlusion regions in a semi-supervised way without any post-processing. We compare our method to the current existing state-of-the-arts methods and shows our results to be performed both qualitatively and quantitatively superior.