Improving cross-dimensional weighting pooling with multi-scale feature fusion for image retrieval

Abstract In this paper, we aim to achieve effective image representation for image retrieval in an unsupervised manner. To this end, we propose a fully cross-dimensional weighting pooling (FCroW) method to improve the weight strategy of the cross-dimensional weighting pooling (CroW). More specifically, FCroW weights both the non-zero parts and zero-parts of convolutional layers, aiming to obtain robust image representations. In particular, we aggregate multi-scale features extracted by convolutional neural networks using the proposed FCroW, taking into account multiple aspects of visual features captured by the networks. Different weights can be assigned to the features extracted by different layers of the networks. To reduce the effort for parameter tuning, we propose an initial strategy to prune the searching space of the weights, which is achieved by designing constraint rules based on the prior knowledge on relations between the layers of the networks. Based on this, we propose weighted multi-layer feature fusion for similar image representations. Extensive experiments conducted on four public real-world datasets demonstrate the effectiveness of the proposed FCroW method and the pruning strategy for image retrieval.