Data-driven taxonomy forest for fine-grained image categorization

Fine-grained image categorization must handle huge cross-class ambiguities and a large number of classes. Inspired by the success of rigid hierarchical classification, we propose a new flexible hierarchical classification method, called a data-driven taxonomy forest. It constructs a multitude of taxonomies, each of which converts a complex multi-class problem to a more easily tractable path-finding problem. We demonstrate how a stochastic representation of local classification hypotheses incorporated in multiple taxonomies deals skillfully with error propagation and over-fitting. Various strategies for instance space decomposition are investigated from the viewpoint of taxonomy complexity. We comprehensively evaluate our data-driven taxonomy forest using Oxford Flower 102 and Oxford Pet benchmarks and show its superiority in effectiveness and generality to rigid hierarchical classification in fine-grained image categorization tasks.