Guided Deep Metric Learning
Jorge Gonzalez-ZapataIvan Reyes-AmezcuaDaniel Flores-AraizaMauricio Méndez-RuizGilberto Ochoa‐RuizAndres Méndez-Vázquez
6
Citation
43
Reference
10
Related Paper
Citation Trend
Abstract:
Deep Metric Learning (DML) methods have been proven relevant for visual similarity learning. However, they sometimes lack generalization properties because they are trained often using an inappropriate sample selection strategy or due to the difficulty of the dataset caused by a distributional shift in the data. These represent a significant drawback when attempting to learn the underlying data manifold. Therefore, there is a pressing need to develop better ways of obtaining generalization and representation of the underlying manifold. In this paper, we propose a novel approach to DML that we call Guided Deep Metric Learning, a novel architecture oriented to learning more compact clusters, improving generalization under distributional shifts in DML. This novel architecture consists of two independent models: A multi-branch master model, inspired from a Few-Shot Learning (FSL) perspective, generates a reduced hypothesis space based on prior knowledge from labeled data, which guides or regularizes the decision boundary of a student model during training under an offline knowledge distillation scheme. Experiments have shown that the proposed method is capable of a better manifold generalization and representation to up to 40% improvement (Recall@1, CIFAR10), using guidelines suggested by Musgrave et al. to perform a more fair and realistic comparison, which is currently absent in the literature.Keywords:
Representation
Similarity (geometry)
Feature Learning
Manifold (fluid mechanics)
Isomap has attracted attentions recently due to its prominent performance on nonlinear dimensionality reduction.However,how to implement effective learning for data on manifold with rings is still a remaining problem.To solve this problem,a systemic strategy is presented in this study.Based on the intrinsic implementation principle of Isomap,a theorem is presented which gives a sufficient and necessary condition to judge whether a manifold is with rings.Besides,an algorithm for detecting ring structures in the manifold is constructed and a nonlinear dimensionality reduction strategy is developed through polar coordinates transformation.A series of simulation results implemented on a series of synthetic and real-world data sets generated by manifolds with or without rings verify the prominent performance of the new method.
Isomap
Manifold (fluid mechanics)
Cite
Citations (0)
As a new kind of nonlinear dimensionality reduction method,manifold learning is capturing increasing interests of researchers.To understand manifold learning better,the principle is firstly introduced,and then its development history and different representations are summarized,finally several major method are introduced,whose basic thoughts,steps and advantages are pointed out respectively.By the experiments on Swiss-Roll,the selection of neighbors and noise effect are analyzed,the results shows:compared with traditional linear method,manifold learning can discover the intrinsic structure of the samples better.Finally the prospect of manifold learning was discussed for more developments.
Manifold (fluid mechanics)
Manifold alignment
Cite
Citations (0)
This paper proposes a new manifold learning method called "Soinnmanifold". Traditional manifold learning method needs a lot of computation and appropriate priori parameters. This has somewhat restricted the domains in which manifold learning can potentially be applied. However, with the high-dimensional inputs, our method can generate a lowdimensional manifold in the high-dimensional space and determine the intrinsic dimension automatically. Then we will use this manifold to do dimensionality reduction quickly. Experiments demonstrate that our method can get promising results with less time and memory.
Manifold (fluid mechanics)
Manifold alignment
Intrinsic dimension
Cite
Citations (1)
Manifold (fluid mechanics)
Manifold alignment
Isomap
Cite
Citations (10)
Manifold learning based methods have been widely used for non-linear dimensionality reduction (NLDR). However, in many practical settings, the need to process streaming data is a challenge for such methods, owing to the high computational complexity involved. Moreover, most methods operate under the assumption that the input data is sampled from a single manifold, embedded in a high dimensional space. We propose a method for streaming NLDR when the observed data is either sampled from multiple manifolds or irregularly sampled from a single manifold. We show that existing NLDR methods, such as Isomap, fail in such situations, primarily because they rely on smoothness and continuity of the underlying manifold, which is violated in the scenarios explored in this paper. However, the proposed algorithm is able to learn effectively in presence of multiple, and potentially intersecting, manifolds, while allowing for the input data to arrive as a massive stream.
Isomap
Manifold (fluid mechanics)
Streaming Data
Clustering high-dimensional data
Smoothness
Manifold alignment
Cite
Citations (13)
Manifold learning is a new data dimensional reduction method,which can reveal the inherent discipline.Its objective is to discover the low-dimensional manifold structure embedded in high dimensional data space,and give an effective low-dimensional formulating.The current manifold learning method gets more and more attentions in the Pattern Recognition and Machine Learning fields for its outstanding data reduction and visualization capabilities.The basic idea of manifold learning was described and the advantages and disadvantages of these algorithms were analyzed,and the LLE was used to estimate the head pose.A good recognition rate was achieved.
Manifold (fluid mechanics)
Manifold alignment
Dimensional reduction
Cite
Citations (0)
Manifold (fluid mechanics)
Manifold alignment
Ensemble Learning
Cite
Citations (0)
Aiming at the problem of nonlinear dimensionality reduction,presents local linear embedding(LLE)algorithm.The nonlinear structure in high dimensional data space is exploited with the local symmetries of linear reconstructions.Maps the data points in high dimensional space into corresponding data points in lower dimensional space under preserving distance between data points.Introduces a manifold learning algorithm of LLE,summarizes some problems of LLE and its research status,and discusses the prospect of LLE.
Manifold (fluid mechanics)
Data point
Manifold alignment
Data space
Cite
Citations (0)
Manifold learning-based encoders have been playing important roles in nonlinear dimensionality reduction (NLDR) for data exploration. However, existing methods can often fail to preserve geometric, topological and/or distributional structures of data. In this paper, we propose a deep manifold learning framework, called deep manifold transformation (DMT) for unsupervised NLDR and embedding learning. DMT enhances deep neural networks by using cross-layer local geometry-preserving (LGP) constraints. The LGP constraints constitute the loss for deep manifold learning and serve as geometric regularizers for NLDR network training. Extensive experiments on synthetic and real-world data demonstrate that DMT networks outperform existing leading manifold-based NLDR methods in terms of preserving the structures of data.
Manifold (fluid mechanics)
Manifold alignment
Autoencoder
Cite
Citations (3)
A detailed retrospection was made on nonlinear dimensionality reduction methods in manifold learning,whose advantages and defects were pointed out respectively.Compared with traditional linear method,nonlinear dimensionality reduction methods in manifold learning could discover the intrinsic dimensions of nonlinear high-dimensional data effectively,help researcher to reduce dimensionality and analyzer data better.Finally,the prospect of nonlinear dimensionality reduction methods in manifold learning was discussed,so as to extend the application area of manifold learning.
Manifold (fluid mechanics)
Manifold alignment
Diffusion map
Cite
Citations (6)