Inductive t-SNE via deep learning to visualize multi-label images

Abstract This work presents a methodology for dimensionality reduction of images with multiple occurrences of multiple objects, such that they can be placed on a 2-dimensional plane under the constrain that nearby images are similar in terms of visual content and semantics. The first part of this methodology adds inductive capabilities to the well known t-SNE method used for visualization, thus making possible its generalization for unseen data, as opposed to previous extensions with only transductive capabilities. This is achieved by pairing the base t-SNE with a Deep Neural Network. The second part exploits semantic information to perform supervised dimensionality reduction, which results in better separability of the low-dimensional space, this is, it separates better images with no relevance, while retaining the proximity of those images with partial relevance. Since dealing with images having multiple occurrences of multiple objects requires the consideration of partial relevance, additionally we present a definition of partial relevance for the evaluation of classification and retrieval scenarios on images, or other documents, that share contents, at least partially.