To replicate or not to replicate, or when did we start to ignore the concept of statistical power?

Abstract Designing (sensory) studies with maximal efficiency is of critical importance in almost all applications. In addition to the required number of panelists, a common question is whether replications are adding any value to obtain more robust results, or whether their contribution is too limited to justify the effort, as addressed in two recent papers. We argue that the conclusions drawn, namely that replications are usually not needed, are unduly generalized. Two situations that are very similar when replications are employed can be quite different without replications. To determine the power of a study, we need to consider not only the total variability of the data but, more specifically, both the within-panelist and between-panelists components individually. In contrast, comparisons of observed significances cannot offer profound guidance on whether replications are needed. Instead, the concept of statistical power provides much better guidance.