Exploiting Category-Level Semantic Relationships for Fine-Grained Image Recognition

We present a label-based, semantic distance induced regularization learning method for Fine-grained image recognition (FGIR). In contrast to previous label-based methods that involve a nontrivial optimization in multi-task metric learning, our approach can be integrated into an end-to-end network without introducing any extra parameters, thus easy to be optimized. To this end, a category-level hierarchical distance matrix (HDM) that encodes semantic distance between subcategories through a tree-like label hierarchy is constructed. HDM is then incorporated into a DCNN to aggregate misclassified prediction probabilities for model learning, thus providing additional discriminative information for fine-grained feature learning. Experiments on three fine-grained benchmark datasets (Stanford Cars, FGVC-Aircraft, CUB-Birds) validate the effectiveness of our approach and demonstrate its improvements over previous methods.