Mixture factor analysis with distance metric constraint for dimensionality reduction

Abstract Dimensionality reduction (DR) is a key preprocessing stage in high-dimensional data classification. Traditional linear DR algorithms, e.g., Linear Discriminant Analysis, transform the original data into a low-dimensional subspace with a linear transformation matrix. However, these methods cannot handle complex nonlinearly separable data. Although some nonlinear DR methods, e.g., Locally Linear Embedding, are proposed to solve this problem, most of them are unsupervised, which only focus on the data structure hidden in the original high-dimensional space, rather than maximizing the inter-class separability of the transformed data, thus reducing the classification accuracy. To tackle this challenge, a novel supervised nonlinear DR algorithm, distance metric restricted mixture factor analysis (DMR-MFA), is proposed for high-dimensional data classification. In DMR-MFA, the original data is divided into several clusters, and the generation of original data in each cluster is described via a factor analysis model. Meanwhile, the distance metric constraint (DMC) is used for maximizing the separability of transformed low-dimensional data from different classes. Moreover, the optimal model parameters are learned via the joint optimization of log-likelihood function and DMC loss function, which makes the DMR-MFA possible to obtain the more separable low-dimensional embeddings while accurately describing the original data. Experimental results on synthetic data, benchmark datasets and high-resolution range profile data demonstrate that our method can handle nonlinearly separable data and improves the classification accuracy of data with high dimensionality.