Meta-learning with neural networks and landmarking for forecasting model selection an empirical evaluation of different feature sets applied to industry data

Although artificial neural networks are occasionally used in forecasting future sales for manufacturing in industry, the majority of algorithms applied today are univariate statistical time series methods for level, seasonal, trend or trend-seasonal patterns. With different statistical methods created for different time series patterns, large scale applications on 10,000s of times series require automatic method selection, often done manually by human experts based on various time series characteristics, or automatically using error metrics of past performance. However, the task of selecting adequate forecasting methods can also be viewed as a supervised learning problem. For instance, a neural network can be trained as a meta-learner relating characteristic time series features to the ex post accuracy of forecasting methods for each time series. Past research has proposed different sets of time series features for meta-learning including simple statistical or information-theoretic as well as model-based features, but have neglected the use of past forecast errors. This paper studies the predictive accuracy of using different feature sets for a neural network meta-learner selecting between four statistical forecasting models, introducing error-based features (landmarkers) and statistical tests as time series meta-features. A large-scale empirical study on NN3 industry data shows promising results of including error-based feature sets in meta-learning for selecting time series forecasting models.