A Knowledge Optimization-Driven Network With Normalizer-Free Group ResNet Prior for Remote Sensing Image Pan-Sharpening

Multispectral (MS) images play a crucial role in environmental monitoring or ecological analysis for their large scope, quick acquisition, and big data. With the rapid development of technology and increasing demand, very high-resolution MS images have attracted a lot of attention these days. However, due to sensor equipment and the imaging environment, the spatial resolution of MS images is always restricted. With the help of panchromatic images, pan-sharpening is a very important technique to enhance the spatial details of MS images. In this study, we proposed a knowledge optimization-driven pan-sharpening network with normalizer-free group ResNet prior, called PNXnet, which is unfolded from a physical knowledge optimization-driven variational model. We solved the memory overhead brought by the traditional ResNet relying on batch normalization. Results on four sensors show that high quantitative indexes and natural visual effects have verified the reliability of PNXnet. Focusing on the near-infrared (NIR) band where spatial details are hard to be injected, we compared the normalized difference vegetation index (NDVI) generated from the fused results, the estimated NDVI shows a high consistency to the ground truth with $R^{2}$ above 0.91. Besides, we also compared the model generation. Furthermore, low model complexity and quicker computational speed make the daily application of PNXnet possible.