Kernel-Target Alignment based Non-linear Metric Learning

Abstract Distance metric learning aims to learn a measure of the pairwise distance between data instances, which is essential for various machine learning algorithms and applications. However, existing linear metric learning methods based on linear transformations fail to capture nonlinear relationships between instances, while most existing kernel-based metric learning algorithms may ignore the correlation between the kernel function and the target learning task during model selection, which results in a suboptimal selection of kernels. To address the aforementioned issues, we propose a method named Kernel Alignment based Metric Learning with Random Fourier Approximation ( KAML RFA ). Specifically, on one hand, we utilize the random Fourier features to approximate the shift-invariant kernel function for distance metric learning. On the other hand, we attempt to maximize the degree of agreement between the kernel function and the target learning task. In this way, distance metric learning is performed in a discriminative feature space. Compared with those kernel metric learning algorithms whose kernel functions fall into a suboptimal, KAML RFA achieves the optimal kernel function for the target task and improves classification accuracy. We develop an efficient solution to solve the proposed optimization problem. Extensive experiments are conducted on several benchmark datasets including NUS-WIDE-LITE, USPS, MNIST, LETTER and 10 UCI datasets to verify the effectiveness of KAML RFA compared with state-of-the-art methods.