LiCaNext: Incorporating Sequential Range Residuals for Additional Advancement in Joint Perception and Motion Prediction

Autonomous driving can obtain accurate perception and reliable motion prediction with the support of multi-modal fusion. Recently, there has been growing interest in leveraging features from various onboard sensors to enhance the primary stages of autonomous driving. This paper proposes LiCaNext to capture additional accuracy advancements in joint perception and motion prediction while maintaining real-time requirements. LiCaNext is the next version of LiCaNet, which fuses LIDAR data in both bird’s-eye view (BEV) and range view (RV) representations with a camera image. In contrast to LiCaNet, we introduce sequential range residual images into the multi-modal fusion network to further improve performance, with minimal increase in inference time. Employing sequential range residual images has a substantial direct impact on motion prediction and positively influences perception. We provide an extensive evaluation on the public nuScenes dataset. Our experiments show that incorporating sequential range residuals secures significant performance gain, with monotonic progress for a larger number of exploited residuals.