Image and tag retrieval by leveraging image-group links with multi-domain graph embedding

A large number of images are available on online photo-sharing services along with rich meta-data, including tags, groups, and locations, etc. For associating two domains of different modalities, e.g. images and tags, Canonical Correlation Analysis (CCA) and its extended methods are used widely. We employ a more flexible graph embedding method called Cross-Domain Matching Correlation Analysis (CDMCA), which can deal with many-to-many relationships between any number of domains, for associating images, tags, and groups. Experiments of Tag-to-Image and Image-to-Tag retrieval tasks are performed on Flickr images. Tags are represented by feature vectors based on semantic word embedding, which enables the searching of even unseen tags. Groups are collections of images, and they provide the information of image-group links. Our experiments show that image and tag retrieval tasks improve by utilizing the group information as the third domain.