Matrix Variate Restricted Boltzmann Machine based on Canonical Polyadic Decomposition

Restricted Boltzmann Machine (RBM) is an important generative model modeling vectorial data. While applying an RBM in practice to images, the data have to be vectorized. This results in high-dimensional data and valuable spatial information has got lost in vectorization. In particular, a large amount of memory is required by commonly used fully-connected layers, making it hard to use the models on low-end devices and stopping the further increase of the model size. In this paper, we proposed a Matrix-Variate Restricted Boltzmann Machine (MVRBMs) model based on Canonical Polyadic Decomposition (CPD). We demonstrate the capacity of MVDBNs on handwritten digit reconstruction and classification.