Toward Automated Model Building from Video in Computer-Assisted Diagnoses in Colonoscopy

ABSTRACT A 3D colon model is an essential component of a computer-aided diagnosis (CAD) system in colonoscopy toassist surgeons in visualization, and surgical planning and training. This research is thus aimed at developingthe ability to construct a 3D colon model from endoscopic videos (or images). This paper summarizes our on-going research in automated model building in colonoscopy. We have developed the mathematical formulationsand algorithms for modeling static, localized 3D anatomic structures within a colon that can be rendered frommultiple novel view points for close scrutiny and precise dimensioning. This ability is useful for the scenariowhen a surgeon notices some abnormal tissue growth and wants a close inspection and precise dimensioning. Ourmodeling system uses only video images and follows a well- established computer-vision paradigm for image-basedmodeling. We extract prominent features from images and establish their correspondences across multiple imagesby continuous tracking and discrete matching. We then use these feature correspondences to infer the camerasmovement. The camera motion parameters allow us to rectify images into a standard stereo con“guration andcalculate pixel movements (disparity) in these images. The inferred disparity is then used to recover 3D surfacedepth. The inferred 3D depth, together with texture information recorded in images, allow us to construct a 3Dmodel with both structure and appearance information that can be rendered from multiple novel view points.Keywords: Calibration, enhanced reality, image-guided therapy, modeling, visualization