3D Volume Visualization and Screen-based Interaction with Dynamic Ray Casting on Autostereoscopic Display

Augmented reality (AR) is an emerging technology to improve visualization experiences. However, visualizing volume data is limited in existing AR systems due to the lack of an intuitive and precise exploration scheme. In this paper, we present a 3D augmented volume visualization and screen-based interaction method. An autostereoscopic handheld display is utilized to achieve naked-eye 3D perception, and a stereo camera is adopted to track the display’s 6 DoF pose. We implement real-time ray casting with GPU acceleration and enhance the visual experience by defining the dynamic view frustum, clipping interaction and transfer function based on the pose of the handheld display. Our display system achieves real-time rendering and tracking performance with dynamic visual effects, allowing a global overview and detailed visualization of arbitrary clipping planes. We also perform a user study to compare an anatomical landmark annotation task on a 2D environment with our system. The results show a significant reduction in completion time and an improvement in depth perception and comprehension of complex topologies. Furthermore, to illustrate the applicability of our system, we present three volumes from different biological scales.