Optical-inertial tracking with active markers and changing visibility

This paper presents an optical-inertial tracking algorithm with explicit and assured optical marker identification. Active optical markers are sequentially or simultaneously triggered to achieve a maximum in quantity and quality of measurements available for tracking. Markers that appear in a camera image are identified by individual activation and are locally tracked in the 2D-images after initial identification. The 2D-position measurements of the cameras are combined with low latency measurements of acceleration and angular velocity from an inertial measurement unit. An Extended Kalman Filter is used for an ultra-tightly coupled data fusion, that takes advantage of all marker measurements with verified identity. The accurate, low latency tracking is robust with respect to temporary marker occlusions, as needed in applications where a robot is directly controlled with a tracked device. The tracking algorithms are implemented in real-time and verified with a test bed in a medical robotics context.