Discernibility Matrix-Based Ensemble Learning

Ensemble learning is admittedly one main paradigm in machine learning, where multiple individual learners are combined together to obtain better performance by making use of the significant diversity among the models. The source of diversity, however, is included in either samples or attributes in some ensemble methods. The concept of discernibility matrix in rough set theory can yield several different attribute reducts, i.e. a series of attribute subsets selected. The attribute subsets obtained are all satisfactory attribute reduction results and different from each other, which exactly correspond with the diversity required by ensemble learning. In this paper, we embed the discernibility matrix to ensemble learning and propose a discernibility matrix-based ensemble learning algorithm named DMEL in which attribute reduction and learning are fused together. On the one hand, the learning performance can be improved in virtue of attribute reduction. On the other hand, a series of good but different attribute subsets obtained by discernibility matrix can ensure the diversity of ensemble learning from the angles of both samples and attributes. In order to ensure the algorithm more general, the k-means discernibility matrix is put forward to deal with numerical data. Experimental results on real-life data sets demonstrate the effectiveness of the proposed method.