Transfer Learning-Based Image Tagging Using Word Embedding Technique for Image Retrieval Applications

In recent days, social media plays a vital role in day to day life which increases the number of images that are being shared and uploaded daily in public and private networks, it is essential to find an efficient way to tag the images for the purpose of effective image retrieval and maintenance. Existing methods utilize feature extraction techniques such as histograms, SIFT, Local Binary Patterns (LBP) which are limited by their inability to represent images in a better way. We propose to overcome this problem by leveraging the rich features that can be extracted from convolutional neural network (CNN) that have been trained on million images. The features are then fed into an Artificial Neural Net, which is trained on the image features and multi-label tags. The tags predicted by the neural net for an image is mapped on to a word embedding plane from which the most similar words for the given tag is retrieved. Along with this ANN, we include an Object detection neural net, which provides additional tags for the image. The usage of an additional neural net and a word vector model adds to the aspect of Zero-shot tagging i.e., the ability of the tagger to assign tags to images outside its own training class examples. The dataset utilized here is the Flickr-25k dataset.