GPS-ReID: A Benchmark for Cross-Modal Pedestrian Retrieval

Traditional person re-identification aims to retrieve the surveillance images containing the same pedestrian. As the quick development of modern cities, a large number of multimodal personal information, including mobile information and social network log, is available and useful for customer identification and crime tracking. Compared with traditional person reidentification (ReID) only based on image modality, how to make full use of multi-modal information for efficient person ReID is more challenging. In this paper, we propose a brand new cross-modal pedestrian retrieval task based on a novel multi-modal dataset containing GPS trajectories and surveillance images. Three sub-tasks are evolved in the benchmark: unsupervised GPS-to-Image, Image-to-GPS, and Image-to-Image retrieval. In order to further verify our ideas, we propose the Similarity Driven Model (SDM), which utilizes the attention mechanism to improve the performance of domain adaptation. Furthermore, we build a cross-modal heterogeneous graph based on SDM, and adopt Triplet-Walk to uniformly represent different modalities for retrieval. Experimental results demonstrate that our method achieves the state-of-the-art on the GPS-ReID dataset.