Deterministic observer design for vision-aided inertial navigation

The purpose of this tutorial is to report results developed over the last three years for state estimation problems arising with unmanned mobile robots equipped with a monocular camera and a 3-axis gyrometer, complemented with either a velocity sensor, or a 3-axis accelerometer, or an optical flow sensor. Definition and characterization of uniform observability for linear time-varying systems, followed by an observer design framework exploiting first-order approximations of a class of nonlinear systems, are first recalled. The resulting Riccati observers are locally uniformly exponentially stable when associated uniform observability conditions are satisfied. This framework is subsequently applied, with detailed explanations, to a set of practical problems, namely i) classical PnP camera pose estimation using known source points and bearing measurements, ii) the adaptation of this problem to unknown source points by using epipolar constraints, iii) camera pose and velocity estimation using bearing and IMU measurements, and iv) camera velocity and depth estimation using optical flow and IMU measurements from the observation of a planar target. The observer solutions proposed for these last two problems are validated with experimental data.