Adversarial learning for deformable registration of brain MR image using a multi-scale fully convolutional network

Abstract Background and objective Deformable registration is very significant for various clinical image applications. Unfortunately, existing conventional medical image registration approaches, which involve time-consuming iterative optimization, have not reached the level of routine clinical practice in terms of registration time and robustness. The aim of this study is to propose a tuning-free 3D image registration model based on adversarial deep network, and to achieve rapid and high-accurate registration. Methods We propose a fully convolutional network (FCN) to regress the 3D dense deformation field in one shot from the to-be-registered image pair. To precisely regress the complex deformation and produce optimal registration, we design the FCN as a novel multi-scale frame to capture the complementary multi-scale image features and effectively characterize the spatial correspondence between the image pair. Moreover, we learn a discriminator network simultaneously to discriminate the registered two images, where the discrimination loss helps further update the FCN. Thus by the adversarial training strategy, the registration network is urged to produce well-registered two images that are indistinguishable for the discriminator. Results We perform registration experiments on four different brain MR datasets using the model trained by ANDI database. Compared with some state-of-the-art registration algorithms including other newest deeplearning-based methods, the proposed method provides a considerable increase of large than 4% in terms of Dice similarity coefficient (DSC). Moreover, our model also obtains comparable distance errors. More significantly, our model can achieve a high-accurate 3D registration result in average 0.74 s, with roughly hundred speed-up over conventional registration methods. Conclusions The proposed model shows consistent high performance for various registration tasks under a second without any additional parameter tuning, which proves its potential for real-time clinical applications.