Robust feature matching via neighborhood manifold representation consensus

Abstract Feature matching, which aims at seeking dependable correspondences between two sets of features, is of considerable significance to various vision-based tasks. This paper attempts to eliminate false correspondences from given tentative correspondences created on the basis of descriptor similarity. A simple yet efficient approach named neighborhood manifold representation consensus (NMRC) for robust feature matching is presented considering the stable neighborhood topologies of the potential true matches. The core principle of the proposed method is to preserve the local neighborhood structures between two feature points in a potential true match along a low-dimension manifold. Meanwhile, a neighborhood similarity-based iterative filtering strategy for neighborhood construction is designed to improve the matching performance under the circumstance of seriously deteriorated data. The matching problem is further formulated into a mathematical optimization model based on the neighborhood manifold representation and iterative filtering strategy, and a closed-form solution with linearithmic time complexity (i.e., O ( N log N ) ) is derived, which requires only tens of milliseconds to handle over 1000 putative correspondences. Extensive experiments on general feature matching (F-score > 94% for most cases), remote sensing image registration (RMSE