Wide-baseline omni-stereo at junctions: Extrinsic auto-calibration, trajectory and speed estimation

In recent years, there has been an increasing interest for vision-based monitoring of intersections. Most of the works use perspective cameras that work independently. In this paper, we present a wide-baseline stereoscopic system composed of fisheye cameras, perfectly suitable for rural or unsignalized intersections. Accurate extrinsic calibration is required to compute metric information. But the task is quite challenging, because of the wide-baseline and the vegetation. Also, pattern-based methods are hardly feasible without disrupting the traffic. Therefore, we propose a points-correspondence-free solution. Our method is fully-automatic and based on a joint analysis of vehicles motion and appearance. Provided the extrinsic calibration estimated, we achieve trajectory reconstruction with a Kalman filter. The tracking strategy relies on the identification of virtual planes tangent to vehicles bumpers. Extensive experiments are carried out in real conditions with a baseline greater than 22 meters. We achieve accurate auto-calibration at scale with cameras localization error below 28.7 cm (1.3 percent), along with reliable vehicle trajectory estimation and a mean speed error of 3.69 km/h.