Learning Second-order Statistics for Place Recognition based on Robust Covariance Estimation of CNN Features

Abstract Appearance based loop closure detection plays an important role in visual simultaneous localization and mapping systems (vSLAM) by measuring similarity of the places and checking loops to reduce the accumulated error. Traditional loop closure methods execute place recognition by image retrieval with Bag-of-Word model, which forms an orderless representation of local feature descriptors. Convolutional neural networks (CNNs) based features have been investigated for place recognition, where the final descriptors usually are generated by first-order pooling, limiting the representation ability in challenging scenarios. To handle above issue, we introduce high-order statistics into place recognition by developing a novel adaptively normalized covariance pooling method for learning place representations in an end-to-end manner. The proposed method provides robust covariance matrix estimation of high-dimensional and small-size deep features by adaptive covariance normalization (AdaCN). Experimental results on place recognition in the urban environment and image retrieval tasks show that second-order representation is effective, especially for discriminating places with confusing objects, changes in viewpoint and illumination. Besides, the proposed adaptive normalization performs favorably against its counterparts based on Log-Euclidean Riemannian metric and Power-Euclidean metric, while our method is superior to the state-of-the-art place recognition approaches.