Selection of diverse features with a diverse regularization

Abstract Many embedded feature selection methods ignore the correlation among the important features. To reduce correlation, some models introduce constraints to impose sparsity on features, some try to exploit the similarity and group features without changing the objective function. In this paper, we propose diverse feature selection (DFS), which simultaneously performs feature clustering and selection. Given a dataset with known class labels, we separate the features into a set of feature clusters where the features in the same cluster have a higher correlation with each other than with the features in different clusters. A diverse regularization (DR) is proposed to reduce the linear and nonlinear correlations among important features. Using this regularization, DFS can select features that are both informative and diverse. The experimental results on seven image datasets, five gene datasets as well as four other datasets demonstrate the superior performance of DFS.