Multiple Testing of Hypotheses in Biomedical Research

Abstract Multiple Comparisons are standard staple when testing in the environment of Analysis of Variance. The number of experimental conditions or populations is usually small. The assumptions that are needed to be fulfilled for the validity of Analysis of Variance are normally met. Biomedical research is branching away to encompass more complex phenomena. The Genome Wide Association Studies involve a complex disease and thousands of single nucleotide polymorphisms waiting to be explored for a possible association with the disease. Identifying those single nucleotide polymorphisms associated with the disease under focus falls into the realm of multiple testing. Special methods are needed to test thousands of null hypotheses at the same time using data that is correlated across the spectrum of hypotheses. What constitutes a Type I error is an issue. There is no clear understanding what Type II error is. The purpose of this article is to lay out in a systematic fashion basic ideas in multiple testing primarily addressed to biomedical researchers. Parametric and nonparametric procedures in multiple testing are discussed. Several definitions of Type I error are presented and their merits and demerits outlined. The Family Wise Error Rate and False Discovery Rate are discussed at length. A number of multiple testing procedures are presented. A number of examples are presented to highlight the concepts and applications.