TS-Net: Device-Free Action Recognition with Cross-Modal Learning.

Internet of Things (IoT) brings opportunities for wireless sensing and device-free action recognition becomes a hot topic for recognizing human activities. Existing works are trying to fuse WiFi and traditional vision modality in a straightforward way for performance improvement. To overcome the problems such as privacy invasion and computational burden, we design an end-to-end cross-modal learning architecture termed teacher-student network (TS-Net) for device-free action recognition. Different from previous methods with both modalities used for the entire process, our model only use WiFi features without any video information involved during the testing phase. More specifically, we construct a cross-modal supervision scheme in which the visual knowledge and robustness capacity of teacher videos can be transferred into the synchronously collected student wireless signals. The experiments show that our TS-Net can efficiently identify human actions at multi-location without environmental constrains of indoor illumination and occlusion.