Toward high-fidelity haptic interaction with virtual materials: A robotic material scanning, modelling, and display system

We present a robotic setup for the acquisition of object surface material properties. Our setup is able to collect selected kinesthetic characteristics, such as the surface structure and weight, as well as tactile properties like the friction coefficient and the fine roughness characteristics of the object surface. Additionally, the setup captures the visual appearances of the object. The recorded multimodal sensor data provide all relevant information required to form a haptic model of a material sample. We then use this representation in a standard haptic rendering framework and display the virtual materials using an augmented Phantom Omni device. We conducted a subjective experiment which shows that its participants perceived and rated the rendered virtual materials as similar to the corresponding real materials in a direct comparison test where the users interact simultaneously with the real and the virtual material samples. An overall user rating of 4.3 out of 5.0 is achieved during the subjective experiment.