Improved covariant local feature detector

Abstract Local feature detection is a fundamental problem in computer vision. Recently, the research of local feature detection has been switched from handcrafted methods to learning based ones, especially deep learning based ones. A recent successful deep learning based feature detector is the covariant local feature detector that conducts keypoint detection by predicting the transformation of keypoints from nearby pixels. Although this method adopts a new detection framework compared to those methods by computing the keypoint’s likelihood, it treats each pixel equally which may incorrectly detect unstable keypoints. On the other hand, other methods computing the keypoint probability could capture different evidence for keypoint detection as well as provide a natural weight for each prediction in the covariant detector. So, fusing information from other detectors into the covariant detector could improve its performance. Under this motivation, this paper proposes an improved covariant local feature detector by fusing feature information obtained from another detector, which is served as a confidence to guide the voting procedure when converting the predicted transformations into a meaningful score map for keypoint detection. In this way, the fused information can enhance the features that are considered to be good and weaken those unstable features. The proposed method is evaluated on four widely used benchmarks and consistent performance improvement over previous works is observed.