Structured Max-Margin Learning for Inter-Related Classifier Training and Multilabel Image Annotation

In this paper, a structured max-margin learning algorithm is developed to achieve more effective training of a large number of inter-related classifiers for multilabel image annotation application. To leverage multilabel images for classifier training, each multilabel image is partitioned into a set of image instances (image regions or image patches) and an automatic instance label identification algorithm is developed to assign multiple labels (which are given at the image level) to the most relevant image instances. A K-way min-max cut algorithm is developed for automatic instance clustering and kernel weight determination, where multiple base kernels are seamlessly combined to address the issue of huge intra-concept visual diversity more effectively. Second, a visual concept network is constructed for characterizing the inter-concept visual similarity contexts more precisely in the high-dimensional multimodal feature space. The visual concept network is used to determine the inter-related learning tasks directly in the feature space rather than in the label space because feature space is the common space for classifier training and image classification. Third, a parallel computing platform is developed to achieve more effective learning of a large number of inter-related classifiers over the visual concept network. A structured max-margin learning algorithm is developed by incorporating the visual concept network, max-margin Markov networks and multitask learning to address the issue of huge inter-concept visual similarity more effectively. By leveraging the inter-concept visual similarity contexts for inter-related classifier training, our structured max-margin learning algorithm can significantly enhance the discrimination power of the inter-related classifiers. Our experiments have also obtained very positive results for a large number of object classes and image concepts.