Surface reflectance estimation and segmentation from single depth image of ToF camera

Objects in real world can be characterized not just by their shape and color but also by their material characteristic. Estimating surface reflectance of an arbitrary object can play a critical role in object characterization, recognition and realistic reconstruction. We propose a reflectance estimation and segmentation method from single depth image taken by a consumer time of flight (ToF) depth camera. Surface points are grouped and segmented based on the similarity of their estimated reflectance. Experimental results show that our estimated surface reflectance provides better discrimination of different material surfaces. Moreover, reconstructed 3D model of an object can be visualized better with realistic interaction with light source based on the obtained reflectance. HighlightsFirst practical reflectance estimation framework using a mixture of Gaussian model and single consumer ToF depth camera.Semantic image understanding based on varying reflectance characteristics.Reflectance-aware 3D scene rendering for improved visualization of 3D scene.