Constrained Least-Squares Density-Difference Estimation

We address the problem of estimating the difference between two probability den- sities. A naive approach is a two-step procedure thatrst estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because therst step is performed without regard to the second step and thus a small error in therst stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density- difference (LSDD) estimator has been proposed. LSDD directly estimates the den- sity difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and the- oretically prove that CLSDD improves the accuracy of density difference estimation for correctly specied parametric models. The usefulness of the proposed method is also demonstrated experimentally.