Visible-Infrared person re-identification based on frequency-domain simulated multispectral modality for dual-mode cameras

With the prevalence of dual-mode cameras in surveillance systems, visible-infrared person reidentification (VI-ReID) has become an emerging topic. Existing studies of VI-ReID roughly fall into three categories: straightforwardly extracting features, improving loss functions, and conducting visibleinfrared modality generation. The generation methods avoid the shortcoming of the former two that training models are generally vulnerable to parameter changes. However, these generation methods are usually based on spatial domain and are unavoidable to damage the original information of images. To tackle these limitations, we propose a novel frequency-domain simulated multispectral (FSMS) modality and visible-FSMS-infrared collaborative learning. FSMS modality consists of three-channel images generated by a channel-level reconstruction of visible images, primarily based on the nonsubsampled contourlet transform (NSCT) cooperating with a lightweight network. The generation exploits crucial spectral information and edge information contained in frequency domain. Then, we design a multi-modality network to conduct the tri-modality collaborative learning where FSMS modality is utilized as an intermediate, thereby preserving the original spatial structure of images. Additionally, a dynamic-weight tri-modality heterogeneous retrieval (THR) loss and a modality-shared classification (MSI) loss are devised to mine discriminative modality-invariant features. A cross-modality invariant (CMI) constraint for further exploring triplet-wise relationships and an intramodality regularizer for relatively stable convergence are introduced. Finally, experimental results show that our algorithm significantly outperforms the latest state-of-the-arts by 5.7% and 4.4% CMC-1 accuracy on two mainstream benchmark datasets, respectively. And the reasons underlying the observed increase in performance are deeply discussed.