Parameter Estimation for an i.i.d. Model

This chapter is very important for understanding the whole book. It starts with very classical stuff: Glivenko–Cantelli results for the empirical measure that motivate the famous substitution principle. Then the method of moments is studied in more detail including the risk analysis and asymptotic properties. Some other classical estimation procedures are briefly discussed including the methods of minimum distance, M-estimates, and its special cases: least squares, least absolute deviations, and maximum likelihood estimates. The concept of efficiency is discussed in context of the Cramer–Rao risk bound which is given in univariate and multivariate case. The last sections of Chap. 2 start a kind of smooth transition from classical to “modern” parametric statistics and they reveal the approach of the book. The presentation is focused on the (quasi) likelihood-based concentration and confidence sets. The basic concentration result is first introduced for the simplest Gaussian shift model and then extended to the case of a univariate exponential family in Sect. 2.11.