A Sufficient Statistics Construction of Exponential Family Le ́vy Measure Densities for Nonparametric Conjugate Models

Conjugate pairs of distributions over infinite dimensional spaces are prominent in machine learning, particularly due to the widespread adoption of Bayesian nonparametric method- ologies for a host of models and applications. Much of the existing literature in the learning community focuses on processes possessing some form of computationally tractable con- jugacy as is the case for the beta process and the gamma process (and, via normalization, the Dirichlet process). For these processes, conjugacy is proved via statistical machinery tailored to the particular model. We seek to address the problem of obtaining a general construction of prior distributions over infi- nite dimensional spaces possessing distribu- tional properties amenable to conjugacy. Our result is achieved by generalizing Hjort’s con- struction of the beta process via appropriate utilization of sufficient statistics for exponen- tial families.