Improving Shallow Neural Networks via Local and Global Normalization

Convolutional neural networks (CNNs) have achieved great success in computer vision. In general, CNNs can achieve superior performance depend on the deep network structure. However, when the network layers are fewer, the ability of CNNs is hugely degraded. Moreover, deep neural networks often with great memory and calculations burdens, which are not suitable for practical applications. In this paper, we propose Local Feature Normalization (LFN) to enhance the local competition of features, which can effectively improve the shallow CNNs. LFN can highlight the expressive local regions while repressing the unobvious local areas. We further compose LFN with Batch Normalization (BN) to construct an LFBN by two ways of concatenating and adding. LFN and BN are excel at handle local features and global features, respectively. Therefore LFBN can significantly improve the shallow CNNs. We also construct a shallow LFBN-Net by stacking the LFBN and conduct extensive experiments to validate it. LFBN-Net has achieved superior ability with fewer layers on various benchmark datasets. And we also insert the LFBN to exiting CNNs. These CNNs with the LFBN all achieve considerable performance improvement.