Visuo-Haptic Display by Embedding Imperceptible Spatial Haptic Information into Projected Images

Visuo-haptic augmented reality (AR) systems that represent visual and haptic sensations in a spatially and temporally consistent manner are used to improve the reality in AR applications. However, existing visual displays either cover the user’s field-of-view or are limited to flat panels. In the present paper, we propose a novel projection-based AR system that can present consistent visuo-haptic sensations on a non-planar physical surface without inserting any visual display devices between a user and the surface. The core technical contribution is controlling wearable haptic displays using a pixel-level visible light communication projector. The projection system can embed spatial haptic information into each pixel, and the haptic displays vibrate according to the detected pixel information. We confirm that the proposed system can display visuo-haptic information with pixel-precise alignment with a delay of 85 ms. We can also employ the proposed system as a novel experimental platform to clarify the spatio-temporal perceptual characteristics of visual and haptic sensations. As a result of the conducted user studies, we revealed that the noticeable thresholds of visual-haptic asynchrony were about 100 ms (temporal) and 10 mm (spatial), respectively.