Multiple agents’ spatiotemporal data generation based on recurrent regression dual discriminator GAN

Abstract Generative Adversarial Networks (GANs) have proven their capability of generating realistic-looking data and have been widely used in image related and time-series applications. However, generation of multiple agents’ spatiotemporal data remains an unexplored region. In this work, we propose a recurrent regression dual discriminator GAN named R2D2GAN. A novel generator is designed to learn mappings from prior stochastic process to multiple agents’ spatiotemporal data, which is conditioned on spatial configuration of multiple agents only. A classification discriminator and a regression discriminator are proposed to represent different features of spatiotemporal data. The classification discriminator learns to represent spatial and sequential features of each agent. The regression discriminator learns to represent inherent sequential dependency for target agent. To stabilize training of GAN, a min–max game is elaborately designed and new training losses are proposed for dual discriminators and the generator. To validate learning ability of R2D2GAN, we embed it in vehicle trajectory prediction application. Through qualitative and quantitative evaluation, we show that the R2D2GAN is capable of generating realistic-looking multiple agents’ spatiotemporal data with acceptable performance degradation in prediction task.