Image up-sampling using deep cascaded neural networks in dual domains for images down-sampled in DCT domain

Abstract Recent researches show that the high-frequency discrete cosine transform (DCT) coefficients can be estimated from low-frequency DCT coefficients by exploiting the spatial correlations. Hence, images coded by DCT such as JPEG/MJPEG/H.264, etc., can be down-sampled in DCT domain, where the high-frequency information can be accurately restored through image up-sampling. In this letter, we propose a novel deep neural network using the cascaded fully connected layers and convolution layers in dual domains (DCT and spatial domains), in order to restore high-frequency DCT coefficients from observed low-frequency DCT coefficients by exploiting the DCT inter-block and spatial correlations. In the proposed network, many recent techniques are adopted, including residual network in dual domains, batch normalization, denseNet, etc. Experimental results show that the proposed cascaded networks in dual domains significantly outperforms the state-of-the-art DCT up-sampling methods in terms of PSNR (0.63–2.57 dB gain), SSIM values, and subjective evaluations on standard image datasets Set5 and Set14.