Matrix variate deep belief networks with CP decomposition algorithm and its application

Deep belief networks (DBNs) are used in many applications such as image processing and pattern recognition. But the data vectorized resulting in the loss of high-dimensional data and valuable spatial information. The classical DBNs model is based on restricted Boltzmann machines (RBMs) and full connectivity between the visible units and the hidden units. It requires a large number of parameters to be trained using an army of training samples. However, it is difficult to obtain so many training samples in reality. To solve this problem, this paper proposes a matrix-variate deep belief networks (MVDBNs) model which is created from matrix variate RBMs whose parameter is restricted as canonical polyadic (CP) decomposition. MVDBNs are composed of two or more matrix-variate restricted Boltzmann machines (MVRBMs) whose input and latent variables are in matrix form. The MVDBNs have much fewer model parameters and deeper layer with better features to avoid overfitting and more accurate model easier to be learned. We demonstrate the capacity of MVDBNs on handwritten digit classification and medical image processing. We also extend MVRBMs to a multimodal case for image super-resolution.