A first approach towards the usage of classifiers’ performance to create fuzzy measures for ensembles of classifiers: a case study on highly imbalanced datasets

In this work we study the possibility of learning fuzzy measures from classifiers’ performance for improving the standard aggregation methods in classifier ensembles. Fuzzy measures are set-valued functions, which are not necessarily additive, and they are the basis for constructing non-linear fuzzy integrals, such as Choquet or Sugeno integral. These integrals have shown to be very useful in the aggregation of interacting criteria, since this interaction can be well modeled by a fuzzy measure. Classifier ensembles are composed of several classifiers and are aimed at improving the performance of every one of their counterparts. There are two main aspects about ensembles, first, how to build them, and second, how to combine the outputs of all their members. In this work, we focus on the second part, which is a key factor to obtain a successful ensemble. More specifically, we focus on the usage of fuzzy measures for the aggregation phase aiming at taking into account the coalitions and interactions among the members of the ensemble. Our hypothesis is that taking such information into account can lead to better performance. Moreover, we propose to directly obtain the fuzzy measure from data by considering the performance of each subset of classifiers in the ensemble. This way, one needs not include any additional learning for the fuzzy measure that can easily lead to overfitting. In order to test the usefulness of the proposed fuzzy measure, we will consider a set of 33 highly imbalanced datasets and we will develop a complete experimental study comparing the proposed combination scheme with other approaches commonly considered in the literature.