Image Inpainting Based on Inside-outside Attention and Wavelet Decomposition

Recently developed deep learning-based image inpainting methods have suggested their potential applications in filling large missing regions displaying plausible content. Nevertheless, most existing studies either merely considered the external information of the missing regions or relied on the region context to yield semantically coherent patches while overlooking the semantic relevance and feature continuity exhibited by missing regions; these demerits are likely to cause a discontinuous contextual structure and blurry texture details. In this work, a novel inside-outside attention layer (IOA) was proposed, capable of exploiting unmasked image features as references as well as learning the affinity between hole features to predict more consistent semantic information. The adversarial loss attributed exclusively on the natural image level cannot adequately generate a sharp texture detail. To address this problem, a texture component discriminator was introduced via wavelet decomposition to enhance the specific performance. Several experiments were performed on the CelebA and Places2 datasets. As revealed from the results, in contrast to the existing research, the proposed method is capable of restoring images with complex structures and significantly enhancing plausible structure and visual quality.