Test Sets to Evaluate Mammography Interpretative Performance: General Principles and On-Field Experience in a Regional Population Screening

Background: Achieving breast screening aims of reducing disease-specific mortality heavily depends on the overall quality of the diagnostic procedures, among which film interpretation is of paramount relevance. Objective: To consider the general difficulties to be overcome in order to create valid mammography test sets, responding to the requisites of being practical to manage, capable of consistently highlighting readers’ shortcomings, and useful as teaching tools. Methods: The method proposed involves utilizing regional data on interval cancer incidence and radiological classification in the setting up of baseline test sets. We also suggest second-line, tailored test sets, adjusted on the individual readers’ data on diagnostic performance, as analyzed by the Regional Reference Centre (RRC) in such detail as the individual readers’ cancer detection rates and interval cancer rates, correlated with the radiopathologic tumor features. As the RRC for Piedmont, we intend to evaluate these tools in the forthcoming months on a Regional basis. Results: Results from this on-field evaluation of nine screening districts within a Regional Project shall be discussed. Conclusion: We believe that a thorough use of both the overall and the individual data on reading performance, as managed by an RRC, may allow optimization of the mammography test set tool, through the development of both baseline sets, primarily aimed to performance evaluation, and tailored test sets, mainly aimed to more effective training. We also believe that many practical difficulties in producing such tools will be substantially overcome in the near future thanks to the availability in digital format of extensive image archives.