Face Frontalization using Hybrid Attention and Relative Average Discriminator based Generative Adversarial Network

Abstract

Facial recognition technology is becoming more widely used in fields such as security,marketing, healthcare, and entertainment. However, in real-world scenario, the acquired facialimages are often non-frontal due to acquisition angles and environmental conditions, results invariability of human faces due to pose and illumination. Side-view faces, due to varyingacquisition angles and environmental factors, often lead to reduced facial recognition accuracy.Therefore, frontalizing these images to a frontal view is crucial. Due to advancement ofGenerative Adversarial Networks (GANs), face frontalization research has shown improvementand enhances performance of face recognition technology. However, these methods stillstruggle with large pose angles above 45?? and do not recover a facial attribute like eyeglass,hair and skin texture. In this paper we proposed face frontalization using combination ofattention and relative average discriminator on GAN (FFRAD-GAN) designed to improvefrontalization of non-frontal faces, especially under large pose angles above 45??. In thegenerator network our proposed approach utilized U-net with combination of attention (self-attention and channel-attention) mechanism. We utilized channel attention in the encodersection, self-attention in the decoder section of the U-net based generator, and relative averagediscriminator in the discriminator block of the network. The proposed approach FFRAD-GANused CAS-PEAL-R1 and Multi-PIE datasets for training and testing, and compared itsperformance with the baseline work. The frontalized facial image quality was evaluated usingRank1-recognition rate, peak signal to noise ratio (PSNR) and Structural similarity index(SSIM). Based on Rank1-recognition rate our FFRAD-GAN model improves the baseline workDMA-GAN from 98.98 to 99.24 on 15??, 96.59 to 97.19 on 30?? and 93.18 to 94.10 on 45??. Basedon SSIM and PSNR our model also got 38.76 and 0.9969 values respectively. The resultsindicated that FFRAD-GAN significantly outperforms other face frontalization approaches likeTP-GAN, GSP-GAN, DA-GAN and DMA-GAN in terms of frontalizing facial images with alarge pose angle above 45?? and producing high quality frontalized facial images. So, FFRAD-GAN have a significant impact on face recognition and surveillance systems that requires high-quality frontal facial images for performing their tasks effectively.