Multi-modal image registration via depth information based on point set matching

Image registration is an important pre-processing operation to perform multi-modal joint analysis correctly. However, registration of images captured by different sensors is a very challenging problem due to the apparent differences of scenes. Traditional Coherent Point Drift method (CPD) is a global registration approach, which strongly relies on the extracted features. In the case of infrared and visible images, registration methods based on edges or points are inappropriate since those features might be significantly different. Fortunately, depth information is more robust feature for multi-modal image pairs. In this paper, we propose an algorithm based on Canny to extract edge of objects. And the regions of interest (ROI) is obtained by depth maps of image pairs in which common features usually successfully implemented by point set registration. Experimental results on real world data demonstrate the effectiveness of the proposed approach, which is superior to the traditional CPD algorithm for multi-modal image registration.