Deep Self-Paced Residual Network for Multispectral Images Classification Based on Feature-Level Fusion

The classification methods based on fusion techniques of multisource multispectral (MS) images have been studied for a long time. However, it may be difficult to classify these data based on a feature level while avoiding the inconsistency of data caused by multisource and multiple regions or cities. In this letter, we propose a deep learning structure called 2-branch SPL-ResNet which combines the self-paced learning with deep residual network to classify multisource MS data based on the feature-level fusion. First, a 2-D discrete wavelet is used to obtain the multiscale features and sparse representation of MS data. Then, a 2-branch SPL-ResNet is established to extract respective characteristics of the two satellites. Finally, we implement the feature-level fusion by cascading the two feature vectors and then classify the integrated feature vector. We conduct the experiments on Landsat_8 and Sentinel_2 MS images. Compared with the commonly used classification methods such as support vector machine and convolutional neural networks, our proposed 2-branch SPL-ResNet framework has higher accuracy and more robustness.