Consistency-Based Feature Selection

Feature selection, the job to select features relevant to classification, is a central problem of machine learning. Inconsistency rate is known as an effective measure to evaluate consistency (relevance) of feature subsets, and INTERACT, a state-of-the-art feature selection algorithm, takes advantage of it. In this paper, we shows that inconsistency rate is not the unique measure of consistency by introducing two new consistency measures, and also, show that INTERACT has the important deficiency that it fails for particular types of probability distributions. To fix the deficiency, we propose two new algorithms, which have flexibility of taking advantage of any of the new measures as well as inconsistency rate. Furthermore, through experiments, we compare the three consistency measures, and prove effectiveness of the new algorithms.