Meta‐Methodology: Conducting and Reporting Meta‐Analyses

The practice of evidence-based medicine requires physicians to be familiar with the most relevant research published in the medical literature. Individual randomized clinical trials are well suited to provide compelling evidence of an intervention’s therapeutic benefit. However, it has become very difficult (arguably impossible) for physicians to read every publication of relevance to their particular specialty. Systematic reviews and meta-analyses have therefore become increasingly important. Systematic reviews are descriptive in nature, and “collate, compare, discuss, and summarize the current results” in a particular field. Meta-analysis goes a step further, providing us with a statistical technique to combine results from multiple individual trials and then use this dataset to conduct a new analysis that we could not conduct on the basis of any of the individual trial’s datasets. Before starting our discussions, two general points are noteworthy. First, while the term meta-analysis is typically used in the literature to refer to the entire process of conducting such an analysis, we believe it does not adequately capture and emphasize the need for methodological rigor in the full array of actions required. Certainly, an analysis is conducted, and the term meta-analysis is entirely appropriate when discussing that segment of the process. However, determining the study reports from which we construct our new dataset, choosing the appropriate analytical model, and presenting our results with scientific and clinical decorum are also critically important. Indeed, the mathematical calculations are the easiest part of the overall process, exemplifying very well that the discipline of statistics is much more than mere number crunching. We therefore believe that the term meta-methodology meaningfully describes the entire process, one part of which is conducting a meta-analysis. Second, as we have noted previously, it is unfortunate that, concurrently with the publication of their results in a high-profile medical journal, some meta-analysts disseminate their findings in the mass media “with a bravado that markedly departs from calm, scientific and clinical discourse, and seemingly with the expectation that the nation’s physicians will change their practice of medicine overnight.” As Turner and colleagues noted, “In the era of sensationalist, sound-bite coverage, clinical science sadly falls very low on the list of points to be covered in the allotted 30 seconds of television coverage,” a point not unknown to meta-analysts who deliberately participate in this circus. Fortunately, many more meta-analysts provide physicians, and hence their patients, with appropriately presented information.