Circular Shape Prior in Efficient Graph Based Image Segmentation to Segment Nucleus

A graph based segmentation approach is proposed in this study to segment nucleus from cytology images. This approach utilizes a novel method applying weighted circular shape prior adaptively in efficient graph based image segmentation. The proposed method was evaluated by segmenting nucleus from two public Pap smear image datasets: ISBI 2014 challenge dataset (945 images) and DTU/Herlev intermediate squamous cell dataset (70 images). Segmentation results of the proposed method outperformed the standard one in terms of Dice similarity coefficient, pixel-based precision and recall, Hausdorff distance, and H t metric. Quantitative measures and visual results indicate that the proposed technique produces better nucleus boundaries.