Learning residual refinement network with semantic context representation for real-time saliency object detection

Abstract Salient object detection (SOD) aims to precisely segment out the most attractive areas in a single image. With the rapid development of deep learning, much effort has been paid to learn an effective representation for SOD from bottom-up or top-down pathways. However, they fail to precisely separate out the whole salient object with fine boundaries due to the repeated subsampling operations such as pooling and striding leading to the loss of fine structures and spatial details. To address these issues, in this paper, we propose a residual refinement network with semantic context features for SOD. First, we design an encoder-decoder structure with side-connections to capture the sharper object boundaries, which can not only gradually recover the spatial details in each feature map from top to down, but also enhance the features at all scales with high-level semantic context information. The semantic context enhanced features are further strengthen by using a set of atrous convolutional filters with multiple atrous rates to encode multi-scale context information. Finally, using the side-output features as input, we develop a recurrent residual module to gradually learn to recover the missing boundary details in the previous coarsely predicted saliency map in a coarse-to-fine manner. Extensive evaluations on six popular SOD benchmark datasets demonstrate leading performance of the proposed approach compared with state-of-the-art methods. Especially, our approach runs in real-time at a speed of 29 fps.