End-to-End Multiple Object Tracking with Siamese Networks

Multiple Object Tracking (MOT) consists of two components: detection and data association. In the popular tracking-by-detection models, these two components are separate: all objects of interest in a frame are detected first, and then associated with the objects in tracked queues using intersection-over-union (IOU) of bounding box and/or appearance matching. Appearance feature (ID embedding) can be extracted from a separate re-identification model or from a joint model with the detection network. In this paper, a joint detection and tracking model with Siamese structure is proposed for MOT. We apply CenterNet, an anchor free object detector for detection. Motion prediction and appearance matching are implemented with the network for association. Experimental results demonstrate the effectiveness of our model. State-of-the-art MOTA of 69.9% on MOT20 is achieved.