Notions in Significance Testing of Hypotheses

In this book often we perform hypothesis testing on the basis of what is defined later as p-value, as opposed to testing at a fixed nominal level. The p-value is sometimes called the ‘observed level of significance’ or the ’significance value’. As a data dependent quantity, it serves as a measure of evidence in favor of or against a certain null hypothesis. The practice of making a judgement on a certain hypothesis with a p-value is often called significance testing. It is good statistical practice to report p-values in any study involving tests of hypotheses, especially in biomedical research. This way, the judgment of the adequacy of evidence and significance of study findings can be left to the experts and decision makers. Fixed level testing is useful, however in a few applications such as statistical quality control, and therefore we will provide fixed-level tests as well when they are available. With the present state of modern computing and statistical software, which allows a practitioner to compute the probability coverage under standard distribution in a matter of seconds, one can hardly find an excuse to do otherwise.