PositNN Framework: Tapered Precision Deep Learning Inference for the Edge

The performance of neural networks, especially the currently popular deep neural networks, is often limited by the underlying hardware. Computations in deep neural networks are expensive, have a large memory footprint, and are power hungry. Conventional reduced-precision numerical formats, such as fixed-point and floating point, are not optimal to represent deep neural network parameters with a nonlinear distribution and small dynamic range. The recently proposed posit numerical format with tapered precision represents small values more accurately than the other formats. In this work, we propose a deep neural network framework, PositNN, that uses the posit numerical format and exact-dot-product operations during inference. The efficacy of the ultra-low precision version of PositNN as compared to other frameworks (which use fixed-point and floating point) is demonstrated on three datasets (MNIST, Fashion MNIST, and CIFAR-10), where an {5-8}-bit PositNN outperforms other {5-8}-bit low-precision neural networks across all tasks.