Occlusion handling of visual tracking by fusing multiple visual clues

In this paper, a robust visual tracking system with occlusion handling is proposed to track the target with real-time performance. The thermal camera, which can observe the heat originated from the target such as the human body or vehicle, can collaborate with the color camera to track the target in the cluttered environment or under occlusion. Unlike the general tracking by using the color camera and the thermal camera, which simply verifies the target hypotheses in these two kinds of image domains, a sampling multiple importance resampling scheme is proposed here to efficiently generate the hypotheses and verify them. The better hypotheses in the color and thermal images are selected to evaluate the sparse appearance representation such that the target under severe occlusion can be identified with real-time performance. Using this resampling scheme, the diversity and the convergency are simultaneously considered by adaptively fusing the hypotheses in the color and thermal images. Moreover, the updating strategy of target image model is designed by estimating the occlusion ratio and the environmental similarity such that the robustness of tracking can be greatly increased. Finally, the proposed approaches have been validated in several scenes to present the tracking performance.