A Multi-resolution Medical Image Fusion Network with Iterative Back-Projection

The aim of medical image fusion is to integrate complementary information in multi-modality medical images into an informative fused image which is pivotal for assistance in clinical diagnosis. Since medical images in different modalities always have great variety of characteristics (such as resolution and functional information), the fused images obtained from traditional methods would be blurred in details or loss of information in some degree. To solve these problems, we propose a novel deep learning-based multi-resolution medical image fusion network with iterative back-projection (IBPNet) in this paper. In our IBPNet, up-projection and down-projection blocks are designed to achieve feature maps alternation between high- and low-resolution images. The feedback errors generated in the alternation process are self-corrected in the reconstruction process. Moreover, an effective combined loss function is designed, which can adapt to multi-resolution medical image fusion task. High spatial resolution magnetic resonance imaging (MRI) images and low spatial resolution color Positron emission tomography (PET) images are used to demonstrate the validation of the proposed method. Experimental results show that our method is superior to other state-of-the-art fusion methods in terms of visual perception and objective evaluation.