Hybrid image super-resolution using perceptual similarity from pre-trained network

Abstract The goal of super-resolution (SR) is to recover a high-resolution (HR) image from its corresponding low-resolution (LR) image. It is an ill-posed problem. Most recent methods are based on external training data. They can reconstruct pleasing HR results, especially when the input patch has a similar counterpart within the training dataset. Other methods are driven by self-similarity and are called internal methods. They can produce visually plausible HR images when the input images contain abundant regular structures. In this paper, we propose a hybrid method for image SR that exploits the complementary advantages of external and internal SR methods. Each input LR patch is first super-resolved using convolutional neural network (CNN) for external SR and self-similarity for internal SR. Then, we calculate the perceptual similarity between the feature representations from the pre-trained VGG network to learn an adaptive weight. Finally, our algorithm automatically selects the optimal method on the basis of the calculated adaptive weight. The experimental results of our visual and quantitative evaluations verify the effectiveness of the proposed method, by comparing it with state-of-the-art methods.