Foundations of statistics

The foundations of statistics concern the epistemological debate in statistics over how one should conduct inductive inference from data. Among the issues considered in statistical inference are the question of Bayesian inference versus frequentist inference, the distinction between Fisher's 'significance testing' and Neyman–Pearson 'hypothesis testing', and whether the likelihood principle should be followed. Some of these issues have been debated for up to 200 years without resolution.It is unanimously agreed that statistics depends somehow on probability. But, as to what probability is and how it is connected with statistics, there has seldom been such complete disagreement and breakdown of communication since the Tower of Babel. Doubtless, much of the disagreement is merely terminological and would disappear under sufficiently sharp analysis. The foundations of statistics concern the epistemological debate in statistics over how one should conduct inductive inference from data. Among the issues considered in statistical inference are the question of Bayesian inference versus frequentist inference, the distinction between Fisher's 'significance testing' and Neyman–Pearson 'hypothesis testing', and whether the likelihood principle should be followed. Some of these issues have been debated for up to 200 years without resolution. Bandyopadhyay & Forster describe four statistical paradigms: '(1) classical statistics or error statistics, (ii) Bayesian statistics, (iii) likelihood-based statistics, and (iv) the Akaikean-Information Criterion-based statistics'. Savage's text Foundations of Statistics has been cited over 15000 times on Google Scholar. It tells the following. In the development of classical statistics in the second quarter of the 20th century two competing models of inductive statistical testing were developed. Their relative merits were hotly debated (for over 25 years) until Fisher's death. While a hybrid of the two methods is widely taught and used, the philosophical questions raised in the debate have not been resolved. Fisher popularized significance testing, primarily in two popular and highly influential books. Fisher's writing style in these books was strong on examples and relatively weak on explanations. The books lacked proofs or derivations of significance test statistics (which placed statistical practice in advance of statistical theory). Fisher's more explanatory and philosophical writing was written much later. There appear to be some differences between his earlier practices and his later opinions. Fisher was motivated to obtain scientific experimental results without the explicit influence of prior opinion. The significance test is a probabilistic version of Modus tollens, a classic form of deductive inference. The significance test might be simplistically stated, 'If the evidence is sufficiently discordant with the hypothesis, reject the hypothesis'. In application, a statistic is calculated from the experimental data, a probability of exceeding that statistic is determined and the probability is compared to a threshold. The threshold (the numeric version of 'sufficiently discordant') is arbitrary (usually decided by convention). A common application of the method is deciding whether a treatment has a reportable effect based on a comparative experiment. Statistical significance is a measure of probability not practical importance. It can be regarded as a requirement placed on statistical signal/noise. The method is based on the assumed existence of an imaginary infinite population corresponding to the null hypothesis.