Design of sparse Bayesian echo state network for time series prediction

Echo state network (ESN) refers to a novel recurrent neural network with a largely and randomly generated reservoir and a trainable output layer, which has been utilized in the time series prediction. In spite of that, since the output weights are computed by the simple linear regression, there may be an ill-posed problem in the training process for ESN. In order to tackle this issue, a sparse Bayesian ESN (SBESN) is given. The proposed SBESN attempts to estimate the probability of the outputs and trains the network through sparse Bayesian learning, where independent regularization priors should be implied to each weight rather than sharing one prior for all weights. Simulation results illustrate that the SBESN model is insensitivity to reservoir size and completely outperforms other models.