Comparison of visual grading and free-response ROC analyses for assessment of image processing algorithms in digital mammography

Objective To compare two methods for assessment of image-processing algorithms in digital mammography: free-response receiver operating characteristic (FROC) for the specific task of microcalcification detection and visual grading analysis (VGA). Methods The FROC study was conducted prior to the VGA study reported here. 200 raw data files of low breast density (Breast Imaging–Reporting and Data System I–II) mammograms (Novation DR, Siemens, Germany)—100 of which abnormal—were processed by four image-processing algorithms: Raffaello (IMS, Bologna, Italy), Sigmoid (Sectra, Linkoping, Sweden), and OpView v. 2 and v. 1 (Siemens, Erlangen, Germany). Four radiologists assessed the mammograms for the detection of microcalcifications. 8 months after the FROC study, a subset (200) of the 800 images was reinterpreted by the same radiologists, using the VGA methodology in a side-by-side approach. The VGA grading was based on noise, saturation, contrast, sharpness and confidence with the image in terms of normal stru...