The Visible Patient: Augmented Reality in the Operating Theater

Minimally invasive surgery represents one of the main evolutions of surgical techniques. However, minimally invasive surgery adds difficulty that can be reduced by computer technology. Indeed, from a patient’s medical image (US, CT, or MRI), virtual reality (VR) and augmented reality (AR) can increase the surgeon’s preoperative and intraoperative vision by providing virtual transparency of the patient. VR consists of the 3D visualization of the anatomical or pathological structures visible in the medical image through direct volume rendering or 3D surface rendering of organs and pathologies extracted and modeled from medical images. This 3D modeling can also be used to plan and simulate preoperatively the surgical procedure without imparting risk for the patient. As we will illustrate, VR represents a first major progress of surgery. AR is an extension of VR that consists of fusing the VR view with the real view of the patient in the same position and shape: thus, the patient becomes virtually transparent. To be efficient, the VR view must be perfectly registered onto the real view provided by the surgeon eye (direct AR) or by minimally access camera (indirect AR). Registration can be rigid or nonrigid and can be manual or automatic, the main goal being today to have accurate nonrigid and automatic registration. This registration can and will be increasingly used in link with robotic systems to automatize part of complex or repetitive surgical gestures. In this chapter, we will illustrate how computer-aided surgery will be an inevitable step in the progress of minimally invasive procedures through several applications and results of such innovations.