On-the-fly scene acquisition with a handy multi-sensor system

We present a scene acquisition system which allows for fast and simple acquisition of arbitrarily large 3D environments. We propose a small device which acquires and processes frames consisting of depth and colour information at interactive rates. This allows the operator to control the acquisition process on the fly. However, no user input or prior knowledge of the scene is required. In each step of the processing, pipeline colour and depth data are used in combination in order to gain from different strengths of the sensors. A novel registration method is introduced that combines geometry and colour information for enhanced robustness and precision. We evaluate the performance of the system and present results from acquisition in different environments.