Continuous Stereo Self-Calibration on Planar Roads

This paper presents an algorithm for continuous online estimation of the twelve degrees-of-freedom (12-DoF) extrinsic calibration of a stereo camera system for an autonomous car. An Extended Kalman Filter (EKF) recursively estimates the stereo camera calibration by tracking salient points in 3D space that are visible in both cameras. All extrinsic parameters of the stereo camera calibration are only observable under translation and two independent rotations of the vehicle. However, when driving on urban, paved roads the vehicle performs only limited pitch or roll movement. An analysis of the Fisher information matrix reveals that in these situations especially the installation height is difficult to estimate. For these scenarios a further measurement model is added to the algorithm that utilizes the homography of salient points. This homography is induced by the planar road surface in consecutive camera images. Recognition of the road surface, when the position and orientation of the cameras is unknown, can be difficult. Therefore, a convolutional neural network (CNN) is developed that segments camera images pixel-wisely into three classes: `road’, ‘static (other)’ and ‘potentially dynamic’. Only salient points that are segmented as ‘road’ are considered for the homography measurement model. Points that are segmented as ‘potentially dynamic’ are not taken into account for the calibration algorithm. The structure of the CNN has been chosen carefully to enable segmentation of camera images on a mid-range GPU on-board of our autonomous vehicle. An evaluation of the extended algorithm, based on a recorded dataset, shows a considerably faster estimation of the installation height.