IoU-guided Siamese region proposal network for real-time visual tracking

Abstract Recently, region proposal network (RPN) has been combined with the Siamese network for tracking and shown excellent accuracy and high efficiency. However, the low correlation between the classification score and localization accuracy in tracking has weakened the performance of the tracking model. In this paper, we propose an IoU-guided Siamese RPN (SiamIG) to address this problem. Specifically, SiamIG predicts the intersection-over-union (IoU) with the ground truth for each regressed anchor by using an IoU predictor. Then the predicted IoU is multiplied with the classification score to compute the final score, which is more suitable for localization accuracy. Furthermore, each anchor in the regression branch is assigned an IoU-based weighting factor such that the tracking accuracy can be further improved. Specifically, anchor boxes with high IoU are given more attention because of the IoU-based weighting factor, which helps the model pay more attention to an anchor box with a high IoU. Experiments demonstrate that the proposed method runs at over 200 fps and achieves up to 2.4%(AUC), 3%(EAO) and 2.3%(AUC) improvements over the baseline on the OTB-100, VOT-2016 and UAV123 datasets respectively.