Multiframe Video Satellite Image Super-Resolution via Attention-Based Residual Learning

Video satellite can generate video image sequences with rich dynamic information, thus providing a new way for monitoring moving objects. However, to maintain high temporal resolution, video satellite images usually sacrifice their spatial resolution. Therefore, super-resolution (SR) plays a vital role in improving the quality of video satellite images. In this article, we propose a multiframe video SR neural network (MVSRnet) for video satellite image SR reconstruction. The proposed MVSRnet consists of three main subnetworks: an optical flow estimation subnetwork (OFEnet), an upscaling subnetwork (Upnet) and an attention-based residual learning subnetwork (ARLnet). The OFEnet aims to estimate low-resolution (LR) optical flow of multiple image frames. Upnet is then constructed to enhance the resolution of both input frames and the estimated LR optical flows. Motion compensation is subsequently performed according to the high-resolution (HR) optical flows. Finally, the compensated HR cube is fed to the ARLnet to generate SR results. Different from existing video satellite image SR methods, the proposed MVSRnet is a multiframe-based method with an attention mechanism, which can merge the motion information among adjacent frames and highlight the importance of extracted features. Experiments conducted on Jilin-1 and OVS-1 video satellite images demonstrate that the proposed MVSRnet significantly outperforms some state-of-the-art SR methods.