Kernel GMM and its application to image binarization
2
Citation
6
Reference
10
Related Paper
Citation Trend
Abstract:
Gaussian mixture model (GMM) is an efficient method for parametric clustering. However, traditional GMM can't perform clustering well on data set with complex structure such as images. In this paper, kernel trick, successfully used by SVM and kernel PCA, is introduced into EM algorithm for solving parameter estimation of GMM, which is so called kernel GMM (kGMM). The basic idea of kernel GMM is to apply kernel based GMM in feature space instead of in input data space. In order to avoid the curse of dimension, the proposed kGMM also embeds a step to automatically select discriminative features in feature space. kGMM is employed for the task of image binarization. Result shows that the proposed approach is feasible.Keywords:
Kernel (algebra)
Discriminative model
Feature vector
We introduce kernel entropy component analysis (kernel ECA) as a new method for data transformation and dimensionality reduction. Kernel ECA reveals structure relating to the Renyi entropy of the input space data set, estimated via a kernel matrix using Parzen windowing. This is achieved by projections onto a subset of entropy preserving kernel principal component analysis (kernel PCA) axes. This subset does not need, in general, to correspond to the top eigenvalues of the kernel matrix, in contrast to the dimensionality reduction using kernel PCA. We show that kernel ECA may produce strikingly different transformed data sets compared to kernel PCA, with a distinct angle-based structure. A new spectral clustering algorithm utilizing this structure is developed with positive results. Furthermore, kernel ECA is shown to be an useful alternative for pattern denoising.
Kernel (algebra)
String kernel
Tree kernel
Kernel smoother
Cite
Citations (293)
Recently, the nonlinear projection trick (NPT) was introduced enabling direct computation of coordinates of samples in a reproducing kernel Hilbert space. With NPT, any machine learning algorithm can be extended to a kernel version without relying on the so called kernel trick. However, NPT is inherently difficult to be implemented incrementally because an ever increasing kernel matrix should be treated as additional training samples are introduced. In this paper, an incremental version of the NPT (INPT) is proposed based on the observation that the centerization step in NPT is unnecessary. Because the proposed INPT does not change the coordinates of the old data, the coordinates obtained by INPT can directly be used in any incremental methods to implement a kernel version of the incremental methods. The effectiveness of the INPT is shown by applying it to implement incremental versions of kernel methods such as, kernel singular value decomposition, kernel principal component analysis, and kernel discriminant analysis which are utilized for problems of kernel matrix reconstruction, letter classification, and face image retrieval, respectively.
Kernel (algebra)
Tree kernel
Cite
Citations (8)
In kernel methods such as kernel principal component analysis (PCA) and support vector machines, the so called kernel trick is used to avoid direct calculations in a high (virtually infinite) dimensional kernel space. In this brief, based on the fact that the effective dimensionality of a kernel space is less than the number of training samples, we propose an alternative to the kernel trick that explicitly maps the input data into a reduced dimensional kernel space. This is easily obtained by the eigenvalue decomposition of the kernel matrix. The proposed method is named as the nonlinear projection trick in contrast to the kernel trick. With this technique, the applicability of the kernel methods is widened to arbitrary algorithms that do not use the dot product. The equivalence between the kernel trick and the nonlinear projection trick is shown for several conventional kernel methods. In addition, we extend PCA-L1, which uses L1-norm instead of L2-norm (or dot product), into a kernel version and show the effectiveness of the proposed approach.
Kernel (algebra)
String kernel
Tree kernel
Kernel smoother
Cite
Citations (50)
Sparse representation classification (SRC) and kernel method have been successfully used in pattern recognition. On account of the limitations of the single kernel function, we proposed multiple kernel sparse classification method in face recognition to improve human face recognition rate. The Power kernel function has a good stability, and the Gaussian kernel function has good practicability. The Power kernel function and Gaussian kernel function are linearly combined. Through the transformation of different kernel space, we effectively extract the nonlinear structure information of the human face. Many experimental results show that the multiple kernel sparse representation classification algorithms that based on Power kernel function and Gaussian kernel function have higher recognition rate than that only using the single kernel sparse representation classification.
Kernel (algebra)
String kernel
Tree kernel
Cite
Citations (0)
In the paper, the formation conditions and the characteristics of kernel functions are researched and analysed which are used in kernel principal component analysis algorithm. Kernel principal component analysis algorithm is a new statistic signal processing technique which can extract nonlinear features of images. Kernel functions are key elements for improving it's performance. A new kernel function-combination kernel function is proposed. The new kernel function combines a local kernel function with a global kernel function. The local kernel is conditionally positive definite kernel which can extract local features of images. The global kernel function is polynomial kernel function which can extract global features of images. So the new kernel function can extract not only local features but also global features of images. The new kernel function is applied in kernel principal component analysis for extracting features of images. The test images are MNIST handwriting digits and ORL face database. Features of images are extracting by kernel principal component analysis firstly. Then performing classification by using linear support vector machines, the effect of the new kernel and that of other kernel on kernel principal component analysis are compared. The experiment results indicate the new kernel function certainly improves the performance of kernel principal component analysis.
Kernel (algebra)
String kernel
Tree kernel
Principal component regression
Cite
Citations (4)
Kernel principal component analysis (KPCA) has become a popular technique for process monitoring in recent years. However, the performance largely depends on kernel function, yet methods to choose an appropriate kernel function among infinite ones have only been sporadically touched in the research literatures. In this paper, a novel methodology to learn a data-dependent kernel function automatically from specific input data is proposed and the improved kernel principal component analysis is obtained through using the data-dependent kernel function in traditional KPCA. The learning procedure includes two parts: learning a kernel matrix and approximating a kernel function. The kernel matrix is learned via a manifold learning method named maximum variance unfolding (MVU) which considers underlying manifold structure to ensure that principal components are linear in kernel space. Then, a kernel function is approximated via generalized Nystrom formula. The effectiveness of the improved KPCA model is confirmed by a numerical simulation and the Tennessee Eastman (TE) process benchmark.
Kernel (algebra)
Principal component regression
Cite
Citations (3)
This paper introduces kernel versions of maximum autocorrelation factor (MAF) analysis and minimum noise fraction (MNF) analysis. The kernel versions are based upon a dual formulation also termed Q-mode analysis in which the data enter into the analysis via inner products in the Gram matrix only. In the kernel version, the inner products of the original data are replaced by inner products between nonlinear mappings into higher dimensional feature space. Via kernel substitution also known as the kernel trick these inner products between the mappings are in turn replaced by a kernel function and all quantities needed in the analysis are expressed in terms of this kernel function. This means that we need not know the nonlinear mappings explicitly. Kernel principal component analysis (PCA), kernel MAF, and kernel MNF analyses handle nonlinearities by implicitly transforming data into high (even infinite) dimensional feature space via the kernel function and then performing a linear analysis in that space. Three examples show the very successful application of kernel MAF/MNF analysis to: 1) change detection in DLR 3K camera data recorded 0.7 s apart over a busy motorway, 2) change detection in hyperspectral HyMap scanner data covering a small agricultural area, and 3) maize kernel inspection. In the cases shown, the kernel MAF/MNF transformation performs better than its linear counterpart as well as linear and kernel PCA. The leading kernel MAF/MNF variates seem to possess the ability to adapt to even abruptly varying multi and hypervariate backgrounds and focus on extreme observations.
Kernel (algebra)
String kernel
Cite
Citations (141)
This paper reviewed the classical principal components analysis methods for multivariate data analysis and feature extraction in pattern classification. A kernel-based extension to the classical PCA models was discussed to cope with nonlinear data dependencies. Kernel PCA was implicitly performing a linear PCA in some high-dimensional kernel feature space that was nonlinearly related to input space by using a suitable nonlinear kernel function mapping. And then, the conjunction of kernel PCA method and RBF neural networks was proposed in practical and algorithmic considerations. Finally, we illustrate the usefulness of kernel PCA algorithms by discussing kernel PCA RBF neural networks application in handwritten digit classification
Kernel (algebra)
String kernel
Cite
Citations (3)
Tree kernel
Kernel (algebra)
String kernel
Multiple kernel learning
Cite
Citations (16)
The kernel matrix used in kernel methods encodes all the information required for solving complex nonlinear problems defined on data representations in the input space using simple, but implicitly defined, solutions. Spectral analysis on the kernel matrix defines an explicit nonlinear mapping of the input data representations to a subspace of the kernel space, which can be used for directly applying linear methods. However, the selection of the kernel subspace is crucial for the performance of the proceeding processing steps. In this paper, we propose a component analysis method for kernel-based dimensionality reduction that optimally preserves the pair-wise distances of the class means in the feature space. We provide extensive analysis on the connection of the proposed criterion to those used in kernel principal component analysis and kernel discriminant analysis, leading to a discriminant analysis version of the proposed method. Our analysis also provides more insights on the properties of the feature spaces obtained by applying these methods.
Kernel (algebra)
String kernel
Feature vector
Principal component regression
Optimal discriminant analysis
Cite
Citations (0)