Multi-frame co-saliency spatio-temporal regularization correlation filters for object tracking

Abstract The spatial regularization weight of the correlation filter is not related to the object content and the model degradation in the tracking process. To solve this problem, a new multi-frame co-saliency spatio-temporal regularization correlation filters (MCSRCF) is proposed for visual object tracking. To the best our knowledge, this is the first application of co-saliency regularization to CF-based tracking. In MCSRCF, grayscale features, directional gradient histogram (HOG) features and CNN features are extracted to improve the tracking precision of the tracker. Secondly, the three-dimensional spatial saliency and semantic saliency are introduced to obtain the initial weight of the spatial regularization with object content information. Then, the heterogeneous saliency fusion method is exploited to add a co-saliency spatial regularization term to the objective function to make the spatial penalty weight learn the change of the object region. In additional, the temporal saliency regularization is introduced to learn the information between adjacent frames, which reduces the overfitting effect caused by inaccurate samples. A variety of evaluations are conducted on public benchmarks, and the experimental results show that the proposed tracker achieves good robustness against many state-of-the-art trackers in various complex scenarios.