Methodology for filtering of depth maps based on the MCbR algorithm supported by color, shape and neighboring features

Abstract Depth maps from Time-of-Flight sensors are often noisy with regions of unknown depth denoted as black pixels. Although acceptable inpainting of depth maps using single images can be achieved with non-linear techniques, there are regions whose information is inevitably lost. Modified closing by reconstruction algorithm is a clear example of this. Because of the nature of the holes’ regions, a series of rules to classify if a hole was correctly filled based on its contours pixels is proposed. Then, a method for total filtering based on shape, color, and neighboring characteristics to infer the missing depth values is presented, evaluated and compared qualitatively and quantitatively with other state-of-the-art methods using images from the Middlebury dataset with good results.