Face Frontalization Network Based on Auxiliary Task and Transformer

The existing face frontalization methods only use profile images to generate frontal images, which can lead to problems such as poor generation results and overfitting. In this regard, an auxiliary task generative adversarial network（AT-GAN） with Transformer was proposed. AT-GAN used multi-task correlation to improve the effect of face frontalization and generalization. The main task was face frontalization, and the corresponding frontal faces were generated by using profile faces; the secondary task was to generate corresponding frontal portraits from profile portrait sketches, and to guide and assist the main task, so as to accelerate the convergence of network. The network weights were shared by two tasks, and the feature interaction module based on the visual Transformer was used to deeply integrate the two parts of the features, so as to improve the overall performance of the network and generated the more real frontal portraits. AT-GAN consisted of a generator and a discriminator. The feature extraction part combined key points of face with spatial attention to ensure the accurate extraction of key features by the model. The experimental results show that the Rank-1 recognition rate of AT-GAN on the MASFD and CAS-PEAL-R1 datasets is respectively increased 4.42% and 1.30%, and the visual effect and model generalization are improved.