LightweightDeRain: learning a lightweight multi-scale high-order feedback network for single image de-raining

In recent years, deep convolutional neural networks have gained significant performance in single image de-raining. However, this progress is contributing to their complicated model design. These complicated models generally contain a huge number of parameters, resulting in high memory footprints and low efficiency. To handle this issue, we propose a novel Lightweight Multi-scale High-order Feedback Network (LMHFNet) for single image de-raining. First, we regard the de-raining problem as a multi-stage task and combine a high-order feedback mechanism with global residual learning to assist the network training. This combination brings obvious performance improvement and avoids increasing additional parameters. Then, we design a novel Lightweight Multi-scale (LM) block as the core component of our network by utilizing the depthwise separable convolution. Next, we propose a novel Lightweight Multi-scale ConvLSTM (LM-ConvLSTM) module to integrate the deep features generated by the feedback mechanism. Last, we discuss the influence of different factors (i.e., loss function and network input/output) to tap the maximum potential of our lightweight network. Our LMHFNet could achieve competitive performance compared with the latest state-of-the-art methods (i.e., RCDNet and DRDNet), and bring a 28- or 46- times compression at the same time. The extensive experiments demonstrate the effectiveness and efficiency of our model in both quantitative assessments and visual quality.