Finding a good initial configuration of parameters for restricted Boltzmann machine pre-training

Restricted Boltzmann machines (RBMs) have been successfully applied in unsupervised learning and image density-based modeling. The aim of the pre-training step for RBMs is to discover an unknown stationary distribution based on the sample data that has the lowest energy. However, conventional RBM pre-training is sensitive to the initial weights and bias. The selection of initial values in RBM pre-training will directly affect the capabilities and efficiency of the learning process. This paper uses principal component analysis to capture the principal component directions of the training data. A set of initial parameter values for the RBM can be obtained by computing the same reconstruction of the data. Experiments on the Yale and MNIST datasets show that the proposed method not only retains a strong learning ability, but also significantly accelerates the learning speed.