Face Beauty: Improving Quality of Face with Semantic Segmentation Prior and Style Encoder

Despite image super-resolution has great progress in recent years, state-of-the-art face super-resolution still has much potential to promote visual quality. Most of these methods utilize a deep convolutional neural network to explore a mapping between low resolution and high resolution, but it cannot well explore facial structures and local knowledge. In this work, we propose a face hallucination method that explicitly incorporates semantic segmentation prior and style encoder to improve the quality of low resolution face images. To enhance the feature mapping and color mapping of the face, we focus on transferring the prior information extracted from the segmentation mask to the super-resolution process. Furthermore, we add the color attention residual block as a color fidelity unit to preserve the color information of the mapped area. In this way, we can input the latent code generated by the style encoder as parameters into the network to improve the image quality. Experimental results demonstrate that our proposed model achieves superior performance over state-of-the-art approaches including the enhanced super-resolution generative adversarial networks (ESRGAN) and Residual Channel Attention Networks (RCAN).