XSepConv: extremely separated convolution for efficient deep networks with large kernels

Due to the increasing demand for deploying CNNs on resource-constrained platforms, efficient neural networks are becoming more and more popular, in which depthwise convolution plays an indispensable role. Recently, larger kernel sizes (≥5) have been applied to depthwise convolution, but with significantly increased computational cost and parameter size. In this paper, we propose a novel extremely separated convolutional block (XSepConv), which fuses spatially separable convolutions into depthwise convolution to substantially reduce both the computational cost and the parameter size induced by large kernels. We also propose an extra 2×2 depthwise convolution coupled with a new and improved symmetric padding strategy to compensate for the side effect brought by spatially separable convolutions. XSepConv is a more efficient alternative to vanilla depthwise convolution with large kernel sizes; moreover, extensive experiments on popular image classification and object detection benchmarks demonstrate that XSepConv can strike a better trade-off between accuracy and efficiency, and the improvement is more significant for larger kernels.