Camera model compensation for image integration of time-of-flight depth video and color video

In this paper, we describe a consideration of a method of a camera calibration for TOF depth camera in the case of using with color video camera to combine their images into colored 3D models of a scene. Mainly, there are two problems with the calibration to combine them. One is stability of the TOF measurements, and another is deviation between the measured depth values and actual distances that are based on a geometrical camera model. To solve them, we propose a calibration method for it. At first, we estimate an optimum offset distance and intrinsic parameters for the depth camera to match both the measured depth value and its ideal one. Using estimated offset to consecutive frames and compensating the measured values to the actual distance values each frame, we try to remove the difference of camera models and suppress the noise as temporal variation For the estimation, we used the Zhang’s calibration method for the intensity image from the depth camera and the color video image of a chessboard pattern. Using this method, we can get the 3D models which are matched between depth and color information correctly and stably. We also explain effectiveness of our approach by showing several experimental results.