ESTIMATORS FOR UNNORMALIZED STATISTICAL MODELS BASED ON SELF DENSITY RATIO

A wide family of consistent estimators is introduced for unnormalized statistical models. They do not need normalization of the probability density function (PDF) because they are based on the density ratio between the same PDF at different points; the multiplicative normalization constant is canceled there. We construct a family of estimators based on pair-wise comparison of density ratio and derive several estimators as its special cases. The family includes score matching as its parameter limit and outperforms score matching for the optimal value of the parameter. We share the idea of random transformations with contrastive divergence whereas we do not assume Markov chain and obtain consistent deterministic estimators by analytic averaging.