Optimal Design of Surveys and Experiments

(ProQuest: ... denotes formulae omitted.)1 IntroductionThe main tools of experimental research in sociology and psychology is the theory of surveys and experiments as parts of Mathematical Statistics. Mathematical Statistics developed on the fundament of Probability Theory from the end of 19th century on. At the beginning of the 20th century, Karl Pearson and Sir Ronald Aylmer Fisher were notable pioneers of this new discipline. Fisher's book (1925) was a milestone providing experimenters such basic concepts as his well-known maximum likelihood method and analysis of variance as well as notions of sufficiency and efficiency.When we, in the sequel, speak about experiments, we understand this in the broader sense including also surveys - but see for the fundamental differences of experiments and surveys from the theory of science' point of view for instance Rasch, Kubinger, and Yanagida (2011). In concrete applications, the experiment first has to be planned, and after the experiment is finished, the analysis has to be carried out. We deal in this paper with the pre-experimental phase, i.e. the optimal planning of an experiment.Experimental designs originated in the early years of the 20-th century mainly in agricultural field experimentation. A centre was Rothamsted Experimental Station near London, where Sir Ronald Aylmer Fisher was head of the statistical department (since 1919). There he wrote one of the first books about statistical design of experiments (Fisher, 1935); a book which was fundamental, and promoted statistical technique and application.Everything presented in the following is, however, also very important and applicable in psychological research. The mathematical justification of the methods is not stressed, here, and proofs will be often barely sketched, rather omitted. Readers interested in this are referred to Rasch and Schott (2018).Fisher (1935) also outlined the problem of "Lady tasting tea", now a famous design of a statistical randomized experiment which uses Fisher's exact test and is the original exposition of Fisher's notion of a null hypothesis.We refer in the following first to Fisher's problem, that deals with soil fertility. Because soil fertility in fields varies enormously, a field is partitioned into so-called blocks (or strata in surveys) and each block subdivided into plots. It is expected that the soil within the blocks is relatively homogeneous so that the differences in the yield of the varieties planted at the plots of one block are suggested to be only due to the varieties but not due to soil differences. To ensure homogeneity of soil within blocks, the blocks must not be too large. On the other hand, the plots must be large enough so that harvesting (mainly with machines) is possible. Consequently, only a limited number of plots within the blocks is possible and only a limited number of varieties within the blocks can be tested. If all varieties can be tested in each block, we speak of a complete block design. The number of varieties is often larger than the number of plots in a block. Therefore incomplete block designs were developed, chiefly among them completely balanced incomplete block designs, ensuring that all yield differences of varieties can be estimated with equal variance using models of the analysis of variance. How all this is applicable in psychological research is shown in Rasch, Kubinger, and Yanagida (2011 ).The Experimental Designs originally developed in agriculture soon were used in medicine, in psychology and in engineering or more general in all empirical sciences. Varieties were generalized to treatments, and plots to experimental units. But even today the number v of treatments or the letter y (from yield) in the models of the analysis of variance recall us to the agricultural origin.Experimental designs are an important part in the planning (designing) of experiments. The main principles are (the three R-s):1. …