Preliminary application of mxed reality in neurosurgery: Development and evaluation of a new intraoperative procedure

Abstract During neurological surgery, neurosurgeons have to transform the two-dimensional (2D) sectional images into three-dimensional (3D) structures at the cognitive level. The complexity of the intracranial structures increases the difficulty and risk of neurosurgery. Mixed reality (MR) applications reduce the obstacles in the transformation from 2D images to 3D visualization of anatomical structures of central nervous system. In this study, the holographic image was established by MR using computed tomography (CT), computed tomography angiography (CTA) and magnetic resonance imaging (MRI) data of patients. The surgeon's field of vision was superimposed with the 3D model of the patient's intracranial structure displayed on the mixed reality head-mounted display (MR-HMD). The neurosurgeons practiced and evaluated the feasibility of this technique in neurosurgical cases. We developed the segmentation image masks and texture mapping including brain tissue, intracranial vessels, nerves, tumors, and their relative positions by MR technologies. The results showed that the three-dimensional imaging is in a stable state in the operating room with no significant flutter and blur. And the neurosurgeon’s feedback on the comfort of the equipment and the practicality of the technology was satisfactory. In conclusion, MR technology can holographically construct a 3D digital model of patient's lesions and improve the anatomical perception of neurosurgeons during craniotomy. The feasibility of the MR-HMD application in neurosurgery is confirmed.