Unrotating images in laparoscopy with an application for 30° laparoscopes

In laparoscopic surgery, 30 degree laparoscopes are used to improve the viewing range by rotating the scope, effectively changing the viewing direction. In this case, the camera has to be kept fixed so that the captured video does not rotate as well. As both the surgeon and the assistant use both their hands during surgery, this means that one of them has to interrupt his task in order to rotate the scope against the camera. We propose a method to detect and undo rotations of the image automatically and in real-time, thereby allowing to perform the same action with only one hand. To achieve this, we first recover the camera rotation using Nister’s five-point algorithm within a robust Random Sampling Consensus (RANSAC) estimation, based on Speeded Up Robust Features (SURF). In contrast to traditional pose estimation and localization, we can disregard any translational movement, which enables us to recover the rotational pose even for small baselines. A non-linear refinement step that is adapted to the expected small baseline is performed. Because of smoke, blood, and sudden movements, the pose estimation might get lost. We investigate using the extracted features to recover the pose with a bag-of-visual-words approach based on Integrated Region Matching (IRM). The pipeline is capable of running at above 15 Hz using a PC with a modern graphics board if features are not lost. We present results based on real surgery videos to demonstrate the robustness and efficiency of the system.