Support vector machine ensembles for discriminant analysis for ranking principal components

The problemof ranking linear subspaces in principal component analysis (PCA), for multi-class classification tasks, has been addressed by building support vector machine (SVM) ensembles and AdaBoost.M2 technique. This methodology, named multi-class discriminant principal components analysis (Multi-Class.M2 DPCA), is motivated by the fact that the first PCA components do not necessarily represent important discriminant directions to separate sample groups. The Multi-Class.M2 DPCA proposal presents fundamental issues related to the weakening methodology, parametrization, strategy for SVM bias, and classification versus reconstruction performance. Also, it is observed a lack of comparisons between Multi-Class.M2 DPCA and feature weighting techniques. Motivated by these facts, this paper firstly presents a unified formulation to generate weakened SVM approaches and to derive different strategies of the literature. These strategies are analyzed within Multi-Class.M2 DPCA methodology and its parametrization to realize the best one for ranking PCA features in face image analysis. Moreover, this work proposes variants to improve that Multi-Class.M2 DPCA configuration using strategies that incorporate SVM bias and sensitivity analysis results. The obtained Multi-Class.M2 DPCA setups are applied in the computational experiments for both classification and reconstruction problems. The results show that Multi-Class.M2 DPCA achieves higher recognition rates using less PCA features, as well as robust reconstruction and interpretation of the data.