TactileVR: Integrating Physical Toys into Learn and Play Virtual Reality Experiences

We present TactileVR, a proof-of-concept virtual reality system in which a user is free to move around and interact with physical objects and toys, which are represented in the virtual world. By integrating tracking information from the head, hands and feet of the user, as well as the objects, we infer complex gestures and interactions such as shaking a toy, rotating a steering wheel, or clapping your hands. We create educational and recreational experiences for kids, which promote exploration and discovery, while feeling intuitive and safe. In each experience objects have a unique appearance and behavior e.g. in an electric circuits lab toy blocks serve as switches, batteries and light bulbs.We conducted a user study with children ages 5–11, who experienced TactileVR and interacted with virtual proxies of physical objects. Children took instantly to the TactileVR environment, intuitively discovered a variety of interactions, and completed tasks faster than with non-tactile virtual objects. Moreover, the presence of physical toys created the opportunity for collaborative play, even when only some of the kids were using a VR headset.