Multimodal medical image fusion based on nonsubsampled shearlet transform and convolutional sparse representation

Multimodal medical image fusion technology can assist doctors diagnose diseases accurately and efficiently. However the multi-scale decomposition based image fusion methods exhibit low contrast and energy loss. And the sparse representation based fusion methods exist weak expression ability caused by the single dictionary and the spatial inconsistency. To solve these problems, this paper proposes a novel multimodal medical image fusion method based on nonsubsampled shearlet transform (NSST) and convolutional sparse representation (CSR). First, the registered source images are decomposed into multi-scale and multi-direction sub-images, and then these sub-images are trained respectively to obtain different sub-dictionaries by the alternating direction product method. Second, different scale sub-images are encoded by the convolutional sparse representation to get the sparse coefficients of the low frequency and the high frequency, respectively. Third, the coefficients of the low frequency are fused by the regional energy and the average $${L}_{1}$$ norm. Meanwhile the coefficients of the high frequency are fused by the improved spatial frequency and the average $${l}_{1}$$ norm. Finally, the final fused image is reconstructed by inverse NSST. Experimental results on serials of multimodal brain images including CT,MR-T2,PET and SPECT demonstrate that the proposed method has the state-of-the-art performance compared with other current popular medical fusion methods whatever in objective and subjective assessment.