Instance segmentation in urban scenes using inverse perspective mapping of 3D point clouds and 2D images

The instance segmentation in urban scenes is an important task in smart car applications. Recently, a variety of neural network-based approaches have been proposed. One of the challenges is that there are various scales of objects in a scene, and it requires the neural network to have a large receptive field to deal with the scale variations. In this paper, we utilize a series of fronto-parallel virtual planes and inverse perspective mapping of an input image to the planes, to deal with scale variations. We use LiDAR data to estimate the ground area of the scene and to define virtual planes. Then, the point cloud is used to filter out false-alarms among the over-detection results generated by an off-the-shelf deep neural network, Mask RCNN. The experimental result showed that the proposed approach outperforms Mask RCNN without preprocessing on a benchmark dataset, KITTI dataset [9].