What is wrong with the meta-analyses on honey and oral mucositis due to cancer therapies?

Abstract Objectives The results of meta-analyses currently represent the highest level of evidence in modern medicine. Taking the example of radiotherapy and radiochemotherapy-induced oral mucositis and the effects of honey, we analysed six meta-analyses on the topic to assess the quality of the meta-analyses. Design We analysed the various meta-analyses in detail and compared whether the authors have correctly included the various trials or not. Results We found that the quality of these meta-analyses was low. Especially the more recent meta-analyses included trials in which radiotherapy was not part of the medical intervention or where substances other than pure honey were used. Conclusions It is impossible to determine the underlying reasons why these meta-analyses were able to pass the peer-review system without the request for adequate improvements prior to publication. According to the literature at least 7% of the included meta-analyses revealed false results, but it was assumed that due to limitations of external validity and to the decreased likelihood of updating positive meta-analyses, the true proportion of false positives in the meta-analysis was likely to be higher. However, it is crucial that when severe flaws in meta-analyses are detected that they be reported or the meta-analyses are withdrawn, otherwise the normal reader will take the results as given.