A new optimization algorithm for non-stationary time series prediction based on recurrent neural networks

Abstract Deep neural network with recurrent structure was proposed in the recent years and has been applied to time series forecasting. Many optimization algorithms are developed under the assumption of invariant and stationary data distributions, which is invalid for the non-stationary data. A novel optimization algorithm for modeling non-stationary time series is proposed in this paper. A moving window and exponential decay weights are used in this algorithm to eliminate the effects of the history gradients. The regret bound of the new algorithm is analyzed to ensure the convergency of the calculation. Simulations are done on short-term power load data sets, which are typically non-stationary. The results are superior to the existing optimization algorithms.