The Diversity of Regression Ensembles Combining Bagging and Random Subspace Method

The concept of Ensemble Learning has been shown to increase predictive power over single base learners. Given the bias-variance-covariance decomposition, diversity is characteristic factor, since ensemble error decreases as diversity increases. In this study, we apply Bagging and Random Subspace Method (RSM) to ensembles of Local Linear Map (LLM)-type, which achieve non-linearity through local linear approximation, supplied with different vector quantization algorithms. The results are compared for several benchmark data sets to those of RandomForest and neural networks. We can show which parameters are of major influence on diversity in ensembles and that using our proposed method of LLM combining RSM we are able to achieve results obtained by other reference ensemble architectures.