On the competitive analysis and high accuracy optimality of profile maximum likelihood

A striking result of Acharya et al. [ADOS17] showed that to estimate symmetric properties of discrete distributions, plugging in the distribution that maximizes the likelihood of observed multiset of frequencies, also known as the profile maximum likelihood (PML) distribution, is competitive compared with any estimators regardless of the symmetric property. Specifically, given n observations from the discrete distribution, if some estimator incurs an error ε with probability at most δ, then plugging in the PML distribution incurs an error 2ε with probability at most [EQUATION]. In this paper, we strengthen the above result and show that using a careful chaining argument, the error probability can be reduced to δ1−c · exp(c'n1/3+c) for arbitrarily small constants c > 0 and some constant c' > 0. The improved competitive analysis leads to the optimality of the PML plug-in approach for estimating various symmetric properties within higher accuracy ε > n−1/3. In particular, we show that the PML distribution is an optimal estimator of the sorted distribution: it is ε-close in sorted ℓ1 distance to the true distribution with support size k for any n = Ω(k/(ε2 log k)) and ε > n−1/3, which are the information-theoretically optimal sample complexity and the largest error regime where the classical empirical distribution is sub-optimal, respectively.