A Stacking Ensemble Model of Various Machine Learning Models for Daily Runoff Forecasting
49
Citation
61
Reference
10
Related Paper
Citation Trend
Abstract:
Improving the accuracy and stability of daily runoff prediction is crucial for effective water resource management and flood control. This study proposed a novel stacking ensemble learning model based on attention mechanism for the daily runoff prediction. The proposed model has a two-layer structure with the base model and the meta model. Three machine learning models, namely random forest (RF), adaptive boosting (AdaBoost), and extreme gradient boosting (XGB) are used as the base models. The attention mechanism is used as the meta model to integrate the output of the base model to obtain predictions. The proposed model is applied to predict the daily inflow to Fuchun River Reservoir in the Qiantang River basin. The results show that the proposed model outperforms the base models and other ensemble models in terms of prediction accuracy. Compared with the XGB and weighted averaging ensemble (WAE) models, the proposed model has a 10.22% and 8.54% increase in Nash–Sutcliffe efficiency (NSE), an 18.52% and 16.38% reduction in root mean square error (RMSE), a 28.17% and 18.66% reduction in mean absolute error (MAE), and a 4.54% and 4.19% increase in correlation coefficient (r). The proposed model significantly outperforms the base model and simple stacking model indicated by both the Friedman test and the Nemenyi test. Thus, the proposed model can produce reasonable and accurate prediction of the reservoir inflow, which is of great strategic significance and application value in formulating the rational allocation and optimal operation of water resources and improving the breadth and depth of hydrological forecasting integrated services.Keywords:
Inflow
Boosting
Gradient boosting
Ensemble forecasting
Ensemble Learning
AdaBoost
We propse a new boosting algorithm that mends some of the problems that have been detected in the so far most successful boosting algorithm, AdaBoost due to Freund and Schapire [FS97]. These problems are: (1) AdaBoost cannot be used in the boosting by filtering framework, and (2) AdaBoost does not seem to be noise resistant. In order to solve them, we propose a new boosting algorithm MadaBoost by modifying the weighting system of AdaBoost. We prove that one version of MadaBoost is in fact a boosting algorithm, and we show how our algorithm can be used in detail. We then prove that our new boosting algorithm can be casted in the statistical query learning model [Kea93] and thus, it is robust to random classification noise [AL88].
Boosting
AdaBoost
Cite
Citations (171)
AdaBoost
Boosting
Benchmark (surveying)
Word error rate
False positive rate
Cite
Citations (137)
Peer to peer lending is famous for easy and fast loans from complicated traditional lending institutions. Therefore, big data and machine learning are needed for credit risk analysis, especially for potential defaulters. However, data imbalance and high computation have a terrible effect on machine learning prediction performance. This paper proposes a stacking ensemble learning with features selection based on embedded techniques (gradient boosted trees (GBDT), random forest (RF), adaptive boosting (AdaBoost), extra gradient boosting (XGBoost), light gradient boosting machine (LGBM), and decision tree (DT)) to predict the credit risk of individual borrowers on peer to peer (P2P) lending. The stacking ensemble model is created from a stack of meta-learners used in feature selection. The feature selection+ stacking model produces an average of 94.54% accuracy and 69.10 s execution time. RF meta-learner+Stacking ensemble is the best classification model, and the LGBM meta-learner+stacking ensemble is the fastest execution time. Based on experimental results, this paper showed that the credit risk prediction for P2P lending could be improved using the stacking ensemble model in addition to proper feature selection.
Boosting
AdaBoost
Gradient boosting
Ensemble Learning
Ensemble forecasting
Cite
Citations (6)
With the AdaBoost algorithm,the Boosting algorithm and its theory are introduced.We summarize the current works about Boosting,which includes the AdaBoost's training error and generalization error,and the extensions of AdaBoost for classification problems as well.
Boosting
AdaBoost
Error Analysis
Cite
Citations (0)
Ensemble learning techniques have achieved state-of-the-art performance in diverse machine learning applications by combining the predictions from two or more base models. This paper presents a concise overview of ensemble learning, covering the three main ensemble methods: bagging, boosting, and stacking, their early development to the recent state-of-the-art algorithms. The study focuses on the widely used ensemble algorithms, including random forest, adaptive boosting (AdaBoost), gradient boosting, extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and categorical boosting (CatBoost). An attempt is made to concisely cover their mathematical and algorithmic representations, which is lacking in the existing literature and would be beneficial to machine learning researchers and practitioners.
Boosting
Gradient boosting
AdaBoost
Ensemble Learning
Categorical variable
Cite
Citations (408)
Boosting
AdaBoost
Cite
Citations (0)
Boosting
AdaBoost
Cite
Citations (149)
Boosting is one of the most representational ensemble prediction methods. It can be divided into two series: Boost-by-majority and Adaboost. This paper briefly introduces the research status of Boosting and one of its serials-AdaBoost, analyzes the typical algorithms of AdaBoost.
Boosting
AdaBoost
Cite
Citations (8)
This chapter contains sections titled: Introduction Hypothesis Boosting Problem Learn Boosting by Majority AdaBoost BrownBoost AdaBoost for Feature Selection Conclusion References
Boosting
AdaBoost
Cite
Citations (0)
AdaBoost is a well known boosting method for generating strong ensemble of weak base learners. The procedure of AdaBoost can be fitted in a gradient descent optimization framework, which is important for analyzing and devising its procedure. Cost sensitive boosting (CSB) is an emerging subject extending the boosting methods for cost sensitive classification applications. Most CSB methods are performed by directly modifying the original AdaBoost procedure. Unfortunately, the effectiveness of most cost sensitive boosting methods are checked only by experiments. It remains unclear whether these methods can be viewed as gradient descent procedures like AdaBoost. In this paper, we show that several typical CSB methods can also be view as gradient descent for minimizing a unified objective function. We then deduce a general greedy boosting procedure. Experimental results also validate the effectiveness of the proposed procedure.
Boosting
AdaBoost
Gradient boosting
Cite
Citations (2)