Joint sparse-collaborative representation to fuse hyperspectral and multispectral images

Abstract Representation based methods to fuse a low spatial resolution hyperspectral image (LS-HSI) and a high spatial resolution multispectral image (HS-MSI) for reconstructing a high spatial resolution hyperspectral image (HS-HSI) have attracted increasing interest in recent years. Existing representation based algorithms only emphasize the sparsity of data, ignoring the collaboration, which may cause fusion performance degradation. In this paper, we develop a novel fusion method based on joint sparse-collaborative representation (SCR) for LS-HSI and HS-MSI. The SCR method consists of three steps: 1) sparse and collaborative dictionaries are learned to extract the spectral information of the given LS-HSI from two perspectives; 2) the turbopixel based segmentation is used for obtaining unfixed-size patches to describe the complex local structure of the HS-MSI; 3) the joint sparse-collaborative representation model is established for patch representing to reconstruct the HS-HSI. Compared with existing representation based strategies, the SCR not only considers the data sparsity, but also preserves the collaboration reflecting correlations among different spectral bands. In addition, the CSR more sufficiently utilizes the context of the given data, relying on unfixed-size patch dividing with adaptive adjustment by the turbopixel based segmentation. Experimental results indicate that the SCR achieves better performance than several state-of-the-art algorithms.