A Meta Classifier by Clustering of Classifiers

To learn any problem, many classifiers have been introduced so far. Each of these classifiers has many strengths (positive aspects) and weaknesses (negative aspects) that make it suitable for some specific problems. But there is no powerful solution to indicate which classifier is the best classifier (or at least a good one) for a special problem. Fortunately the ensemble learning provides us with a powerful approach to prepare a near-to-optimum classifying system for any given problem. How to create a suitable ensemble of base classifiers is the most challenging problem in classifier ensemble. An ensemble vitally needs diversity. It means that if a pool of classifiers wants to be successful as an ensemble, they must be diverse enough to cover the errors of each other. So during creation of an ensemble, we need a mechanism to guarantee the ensemble classifiers are diversity. Sometimes this mechanism is to select/remove a subset of the produced base classifiers with the aim of maintaining the diversity among the ensemble. This paper proposes an innovative ensemble creation named the Classifier Selection Based on Clustering (CSBC). The CSBC guarantees the necessary diversity among ensemble classifiers, using the clustering of classifiers technique. It uses bagging as generator of the base classifiers. After producing a large number of the base classifiers, CSBC partitions them using a clustering algorithm. After that by selecting one classifier from each cluster, CSBC produces the final ensemble. The weighted majority vote method is taken as aggregator function of the ensemble. Here it is probed how the cluster number affects the performance of the CSBC method and how we can choose a good approximate value for cluster number in any dataset adaptively. We expand our studies on a large number of real datasets of UCI repository to reach a decisive conclusion.