Intraobserver agreement in interpretation of digital epiluminescence microscopy

Abstract Background: Although a major problem with the classification of epiluminescence microscopy (ELM) findings is the lack of standard definitions, reproducibility of the criteria proposed has never been investigated. Objectives: Our purposes were (1) to perform a review of four major published classifications to obtain a set of apparently well-defined ELM variables and descriptors and (2) to evaluate the ability of one of us to report consistently ELM findings in melanocytic lesions according to these criteria. Methods: Intraobserver agreememt (with a set of 44 selected descriptors) between two readings of 150 digital ELM images was evaluated with the kappa (ϰ) statistic. Subgroups of descriptors were compared for ϰ value distribution. Results: The median ϰ value for the whole series of descriptors was 0.66. Median ϰ did not vary significantly among the four classification systems (ϰ = 0.61 to 0.67). Agreement was significantly better as to the presence or absence of ELM findings (ϰ range, 0.39 to 1.00; median ϰ, 0.77) compared with agreement as to their distribution (ϰ range, 0.10 to 0.79; median ϰ, 0.47; p = 0.0007) and their width, thickness, and size (ϰ range, 0.06 to 0.83; median ϰ, 0.39; p = 0.0075). Conclusion: Although nothing can be inferred from a single study, descriptors associated with low intraobserver agreement are likely to be inadequately defined.