Awareness Learning for Balancing Performance and Diversity in Neural Network Ensembles

An ensemble learning system requires a set of cooperative modules that solve a task together. It is believed that awareness modules being aware of other modules in an ensemble could build strong cooperations each other. In this paper, two levels of awareness are introduced into negative correlation learning in order to control the differences among the individual modules in an ensemble. At the ensemble level, whether individual modules would learn to be more different to the rest of modules on a given data point would depend on how well the data point had been learned by the ensemble. At the individual level, each individual module would try to learn to be more different to the rest of modules on a given data point only if it had not been far away from the other individuals based on the distances measured by their outputs on the data point. Negative correlation learning with such awareness learning was tested on training different structures of neural network ensembles where the relations between differences and cooperation were analyzed in the learning process.