Stereo Matching Confidence Learning Based on Multi-modal Convolution Neural Networks

In stereo matching, the correctness of stereo pairs matches, also called confidence, is used to improve the dense disparity estimation result. In this paper, we propose a multi-modal deep learning approach for stereo matching confidence estimation. To predict the confidence, we designed a Convolutional Neural Network (CNN), which is trained on image patches from multi-modal data, namely the source image pairs and initial disparity maps. To the best of our knowledge, this is the first approach reported in the literature combining multiple modality and patch based deep learning to predict the confidence. Furthermore, we explore and compare the confidence prediction ability of multiple modality data. Finally, we evaluate our network architecture on KITTI data sets. The experiments demonstrate that our multi-modal confidence network can achieve competitive results while compared with the state-of-the-art methods.