Locally Weighted Ensemble Learning for Regression

The goal of ensemble regression is to combine a set of regressors in order to improve the predictive accuracy. The key to a successful ensemble regression is to complementally generate base models and elaborately combine their outputs. Traditionally, the weighted average of the outputs is treated as the final prediction. This means each base model plays a constant role in the whole data space. In fact, we know the predictive accuracy of each base model varies across different data spaces. In this paper, we develop a dynamic weighted ensemble method from locality which is called Locally Weighted Ensemble. The weight of each base model varies with sample, which is realized by introducing soft-max function into the objective function. Besides, regularization is also included to make the objective function well-posed. The proposed method is evaluated on several UCI datasets. Compared with single models and other ensemble models, our proposed achieves better performance. From the experiments, we also find that the convergence of Locally Weighted Ensemble is fast.