Low-Complexity Scene Understanding Network

Multi-task networks often rely on complex architectures in order to perceive and understand the driving scene. Computationally intensive networks achieve state of the art results at the cost of real-time inference on embedded devices. Our proposed unified solution obtains competitive results on multiple tasks, while targeting an embedded platform. We build upon our previous work of performing low-complexity object detection and bottom point prediction and add semantic and instance segmentation tasks while maintaining 19 FPS on the NVIDIA Jetson TX2 embedded platform. We find that sharing layers between task sub-networks is essential for achieving real-time inference. Due to the task similarity and correlation between object detection, bottom point prediction, semantic segmentation and instance segmentation we find that the individual task performance is not greatly impacted by the reduced computational capacity resulted from sharing layers amongst the task sub-networks.