The impact of blind peer review (team reading) of equivocal/abnormal Papanicolaou smears on inter-reader agreement

OBJECTIVE: Peer review of equivocal/abnormal Papanicolaou smears is a standard internal quality assurance measure in cervical cancer screening laboratories. In 2014, we introduced a new method for peer review involving blinding the identity of the first reader. We present the comparison of different measures of inter-reader agreement in 2013 and 2014 in order to measure the impact of blind peer review on possible cognitive biases in reading. METHODS: All suspect slides are proposed for peer review; a single reader's peer reviews are registered independently, with discordant cases discussed for consensus diagnosis. Since 2014, the first reader's name has been blinded to peers. We computed how frequently the initial diagnosis was changed, and how frequently a single reader influenced the final diagnosis when the initial one was changed. We compared κ of the first reader and that of other readers with final diagnosis. Data from 2013 were compared to those of 2014. RESULTS: The final diagnosis changed in 22.9% of cases (range 16.8%-34.4%) in 2013 and in 26.8% (range 13.9%-35.6%) in 2014 (P = .2). Individual peer diagnoses agreed with final diagnosis in 52.3% (range 41.9%-66.3%) in 2013 and in 50.2% (range 36.0%-65.8%) in 2014 (P = .55). The cytologist having the highest proportion of confirmed diagnoses was also the most influential on final diagnosis, both in 2013 and in 2014. Agreement between first reader and final diagnosis was higher than that of each reader in 2013 and in 2014. CONCLUSIONS: Blinding the identity of the first reader had little or no impact on inter-reader agreement measures in our laboratory.