Methodological issues in assessing reproducibility—A comparative study of various indices of reproducibility applied to repeat ELISA serologic tests for Lyme disease

Abstract This study applies and evaluates a variety of different measures of reproducibility. As an example, repeat enzyme-linked immunosorbent assays (ELISA) for antibodies to Borrelia burgdorferi , the etiologic agent of Lyme disease, are used. These repeat tests were part of the 1991 Quality Control Assessment of provincial laboratories that was carried out by the Laboratory Centre for Disease Control (Ottawa). Twenty-seven sera from cases and non-cases were tested by three laboratories, and two laboratories repeated the tests once. Methodological issues discussed include: different methods of assessing reproducibility in the continuous scale; whether repro ducibility should be assessed with data in continuous or categorical form; problems assessing the reproducibility of data that has been standardized using a calibration-regression line; and problems with external generalizability of reproducibility studies of rare diseases. The authors conclude that the statistical method used to assess the reproducibility of a test must be adjusted to suit individual study designs and data, and the usage of the test.