Mass margins spiculations: agreement between ratings by observers and a computer scheme

This study investigated the agreement between breast mass spiculation levels as rated subjectively by observers and a computer scheme. An image dataset with 1,263 mass regions was selected. First, three experienced observers independently rated subjectively the visualized spiculation levels of these mass regions and classified them into three categories (none/minimal, moderate, and severe/significant). We then developed a computerized scheme to detect mass margins and classify the spiculation levels of the suspected mass regions. The scheme applied a hybrid region growth algorithm to segment the mass regions. An edge map was computed inside a 30-pixel-wide band surrounding the mass boundary contour. The scheme then applied a threshold to convert the edge map into a binary image followed by labeling and detecting line orientation. In the original edge map the scheme computed the average local pixel value fluctuation. In the binary edge map, the scheme computed the ratio between the number of "spiculated" pixels and the number of total pixels inside the band. The scheme also computed mass region conspicuity using the original image. Using these three features, a Bayesian Belief Network (BBN) was built to classify mass regions into one of the three spiculation categories. We compared the inter-observer variation as well as agreement levels between the subjective and computerized ratings. Agreement rates between paired observers ranged from 41.3% to 58.8% (Kappa = 0.136 to 0.309). The agreement between the computer scheme and observers' average rating was 49.2% (Kappa = 0.218). This study demonstrated a large inter-observer variability in subjective rating of mass speculation levels as well as a large difference between the rating results of a computerized scheme and observers. As a result, in an Interactive Computer-Aided Diagnosis (ICAD) environment, CAD-selected reference regions may be considered "very similar" by some observers and "not similar" by others. Hence, improving the selection of actually visually similar reference regions by a computerized scheme remains an important yet unsolved task for ICAD development.