Measuring Observer Performance in Chest Radiology: Some Experiences

All decisions made under conditions of uncertainty have error rates. All meaningful decisions are made under conditions of uncertainty. Can this uncertainty be measured? Can variations in how different observers deal with this uncertainty be ascertained? The ability to measure observer performance in diagnostic imaging was one of the issues that initiated the field of medical decision analysis. This article exemplifies an approach and is worth discussing as a preamble to presenting our long-term project of measuring variations in observer performance. The paper focuses on the interpretation of chest x-ray images, although the principles and findings described can be applied to nearly every radiologic modality and interpretation task.