Nonlinear CNN: improving CNNs with quadratic convolutions

In this work, instead of designing deeper convolutional neural networks, we investigate the relationship between the nonlinearity of convolution layer and the performance of the network. We modify the normal convolution layer by inserting quadratic convolution units which can map linear features to a higher-dimensional space in a single layer so as to enhance the approximability of the network. A genetic algorithm-based training scheme is adopted to reduce the time and space complexity caused by the quadratic convolution. Our method is experimented on classical image classification architectures including VGG-16 Net and GoogLeNet and outperforms the original models on the ImageNet classification dataset. The experimental results also show that better performance of our method can be achieved with a shallower architecture. We notice that VGG-16 model is widely used in popular object detection frameworks such as faster R-CNN and SSD. We adopt our modified VGG-16 model in these frameworks and also achieve improvements on PASCAL VOC2007 and VOC2012 dataset.