Camera Calibration and Video Stabilization Framework for Robot Localization

Two major issues in robot localization are Camera Calibration (CC) and Video Stabilization (VS). The effectiveness of CC is highly provisional based on adjusting settings, image quality, and image gradient. Recent breakthrough methods employ fixed threshold to calculate pixel difference between frames and preset variables, and neglect slope information causing blurring effect for image frame selected in CC phase. Additionally, contemporary optical flow requires expert manual setting of Gaussian pyramid parameters such as sigma, down scale factor, and number of levels, which consume a lot of time and efforts to train and measure. Apart from that, the localization key challenges of humanoid stereo vision are large motion, motion blur, and defocus blurs of image. Though state-of-the-art approaches used landmark recognition and probabilistic models to overcome those issues, yet localization accuracy is still poor due to image distortion. This work proposed a framework for robot localization via CC and VS methods and triangulation concept. The framework with Fuzzy Camera Calibration (FCC) achieved better results in re-projection error compared to s about 0.85 and 2.62 in pairs based on self-collected dataset, whereas FCC versus Ferstl scored approximately 0.21 and 0.24 in pairs using Time-of-flight camera dataset. For VS, this framework with Fuzzy Optical Flow (FOF) method achieved second rank compared to the state-of-the-art methods such as Farneback, Brox (GPU), LK (GPU), Farneback (GPU), Dual_TVL1, and Simple Flow tested on SINTEL benchmark datasets. Finally, our proposed stereo vision, localization framework also outperformed Mono Vision method vision about 4.07 cm and 61.07 cm subsequently of distance errors.