Assessment of change in breast density: reader performance using synthetic mammographic images

A recent study has shown that breast cancer risk can be reduced by taking Tamoxifen, but only if this results in at least a 10% point reduction in mammographic density. When mammographic density is quantified visually, it is impossible to assess reader accuracy using clinical images as the ground truth is unknown. Our aim was to compare three models of assessing density change and to determine reader accuracy in identifying reductions of 10% points or more. We created 100 synthetic, mammogram-like images comprising 50 pairs designed to simulate natural reduction in density within each pair. Model I: individual images were presented to readers and density assessed. Model II: pairs of images were displayed together, with readers assessing density for each image. Model III: pairs of images were displayed together, and readers asked whether there was at least a 10% point reduction in density. Ten expert readers participated. Readers' estimates of percentage density were significantly closer to the truth (6.8%-26.4%) when images were assessed individually rather than in pairs (9.6%-29.8%). Measurement of change was significantly more accurate in Model II than Model I (p<0.005). Detecting density changes of at least 10% points in image pairs, mean accuracy was significantly (p<0.005) lower (58%-88%) when change was calculated from density assessments than in Model III (74%-92%). Our results suggest that where readers need to identify change in density, images should be displayed alongside one another. In our study, less accurate assessors performed better when asked directly about the magnitude of the change.