693 A DESIGN-EVALUATION MODEL FOR DIAGNOSTIC STUDIES

Diagnostic studies may suffer design deficiencies which obfuscate conclusions. A design-evaluation model was developed and validated to identify common errors. Thirty-five original research publications focusing on diagnostic testing were characterized for ultimate validity by independent editorial assessment. Each publication was evaluated for 31 elements of basic design, test technology, data analysis, and clinical application. Six biometricians ranked model items, of which 6 were considered critically important and the remainder conditionally important. Raw publication scores significantly (p<0.001) correlated with subsequent evidence substantiating or refuting study conclusions. In 25 correct publications, the overall model score mean was 0.79 (SD 0.08), as compared to 0.37 (SD 0.04) in 10 with incorrect conclusions. Fourteen individual model items present in the manuscripts were significantly associated with conclusion validity (p<0.01). The independent editorial assessment emphasized 12 of these individual items. This design-evaluation model can supplement expert assistance or personal experience in planning, conducting, or evaluating diagnostic studies. Journals which establish minimum design criteria might avoid publication of articles with potentially invalid conclusions.