Segmentation based Multi-View Stereo

This paper presents a segmentation based multi- view stereo reconstruction method. We address (i) dealing with uninformative texture in very homogeneous image ar- eas and (ii) processing of large images in affordable time. To avoid searching for optimal surface position and orien- tation based on uninformative texture, we (over)segment im- ages into segments of low variation of color and intensity and use each segment to generate a candidate 3D planar patch explaining the underlying 3D surface. Every point of the surface is explained by multiple candidate patches gen- erated from image segments from different images. Observ- ing that the correctly reconstructed surface is consistently generated from different images, the candidates that do not have consistent support by other candidates from other im- ages are rejected. This approach leads to stable and good results since (i) we use larger 3D patches in homogeneous image areas where small patches covered by uninformative texture would lead to ambiguous results, and (ii) we accept only candidates that are consistent across several images. Since the image segmentation used is very fast and it consid- erably reduces the number of candidates per image on typ- ical scenes, we typically generate and test relatively small number of 3D hypotheses per image and thus can process large images in affordable time. We demonstrate the per- formance of our algorithm on large images from Strecha's dataset.