Letter to the Editor regarding Peto T;UK COVID-19 Lateral Flow Oversight Team: COVID-19: Rapid antigen detection for SARS-CoV-2 by lateral flow assay

It is essential that studies of the accuracy of lateral flow antigen tests (LFTs) to detect SARS-CoV-2 are accurately and completely reported and that data are widely available and freely shared to ensure public and health policy are properly informed [1]. The FDA have raised concerns on the evidence for the Innova LFT [2]. Peto report key studies from Porton Down and the University of Oxford which form the evidence base for use of Innova and its rebranded version (the NHS Test-and-Trace LFT) [3]. We highly commend the authors for kindly sharing their data [4] with our Cochrane team for inclusion in our systematic review of the accuracy of rapid antigen tests [5]. However, we have observed discrepancies between the report of these studies in this journal, information presented in their preprint [6], and the data the investigators provided to our team. We request clarification from the authors. 1) The paper computes specificity by combining data on 6954 Innova tests across studies from “negative samples” (Table 2) [3]. This implies all 6954 samples were verified by a reference standard which returned a negative result (anticipated to be RT-PCR, but no reference standard is described). The preliminary report indicated none of the 3985 participants in the school studies contributing to the estimate of specificity received RT-PCR or any other test to verify their disease status [4], and the data sent to the Cochrane team indicated that the 17 participants in the school studies who tested positive on the Innova assay were parallel-tested with RT-PCR [5]. Can the authors clarify which students received a reference standard, and if so, what test was used and when it was done? If full verification of disease status was not performed, computing specificity using a sample size of 6954 (rather than 2969) is not correct. Notably only 1852 of these samples appear to have been tested using dry swabs in line with manufacturer's instructions, so the relevant sample size is likely even smaller. 2) Of the 17 school samples reported to the Cochrane team as Innova positive, 14 were false positives as the RT-PCR test was reported negative [5]. Whilst unable to compute sensitivity or specificity from these studies, knowing the numbers of test positives and false positives allows computation of a positive predictive value of 18% (95% confidence interval 4 to 43%). This important result is not reported in the publication or preprint. (The publication suggests 16 false positives and higher total numbers tested [3], so the exact value may slightly differ). The positive predictive value of school testing has been an important debate [7], particularly given school testing was originally introduced without RT-PCR confirmatory testing, thus its omission from this report is of relevance. Can the authors clarify that this result is correct and explain why it has been excluded from this report? 3) The authors report a specificity of 100% but no sensitivity value for the Armed Forces cohort. A full 2 × 2 table for this study was provided to and included in the Cochrane review – Innova was positive in 13 out of 46 who were PCR positive (sensitivity 28, 95% confidence interval (16 to 43%)) [5]. Why has this result not been reported? The STARD reporting guideline, adopted by the Lancet journals, aims to assist authors, peer reviewers and editors in ensuring that the methods and results of test accuracy studies are fully reported [8]. The reporting criteria have not been fully adhered to by this article. Regarding the evaluation in schools, if only LFD positive results were verified, the study can only estimate test failure rates, the feasibility of testing, and the positive predictive value. Such a design falls short of what is expected from field evaluations of a diagnostic test [1], particularly given that this was the first large scale evaluation of testing asymptomatic individuals. Testing in schools was introduced in the UK without evidence of the sensitivity of the test to detect cases in this population, which these studies could have easily addressed should verification been done [7]. Whilst we have observed large, well reported trials of vaccines and drugs following established methodology during the pandemic, the same level of methodological rigour has not been applied to test evaluations. We have great concern that the evidence supporting the use of Innova for mass testing has not been obtained and reported with adequate attention to the scientific method required to provide a sound evidence base for policy-making. Yours Jonathan J Deeks PhD CStat FMedSci, Professor of Biostatistics, University of Birmingham, UK Jacqueline Dinnes PhD, Senior Researcher, University of Birmingham, UK Clare Davenport MBChB, PhD, FFPH, Clinical Senior Lecturer, University of Birmingham, UK Yemisi Takwoingi DVM, PhD, Professor of Test Evaluation and Evidence Synthesis, University of Birmingham, UK Matthew McInnes Professor of Radiology and Epidemiology, University of Ottawa, Canada Mariska M. G. Leeflang, PhD, Associate professor, University of Amsterdam, Netherlands Jane Cunningham MD, Medical Officer, World Health Organisation, Switzerland on behalf of the Cochrane COVID-19 Diagnostic Test Accuracy Group.