Exponential family

In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, based on some useful algebraic properties, as well as for generality, as exponential families are in a sense very natural sets of distributions to consider. The term exponential class is sometimes used in place of 'exponential family', or the older term Koopman-Darmois family. The terms 'distribution' and 'family' are often used loosely: properly, an exponential family is a set of distributions, where the specific distribution varies with the parameter; however, a parametric family of distributions is often referred to as 'a distribution' (like 'the normal distribution', meaning 'the family of normal distributions'), and the set of all exponential families is sometimes loosely referred to as 'the' exponential family. [ e η 1 ∑ i = 1 k e η i ⋮ e η k ∑ i = 1 k e η i ] {displaystyle {egin{bmatrix}{dfrac {e^{eta _{1}}}{sum _{i=1}^{k}e^{eta _{i}}}}\vdots \{dfrac {e^{eta _{k}}}{sum _{i=1}^{k}e^{eta _{i}}}}end{bmatrix}}} [ e η 1 1 + ∑ i = 1 k − 1 e η i ⋮ e η k − 1 1 + ∑ i = 1 k − 1 e η i 1 1 + ∑ i = 1 k − 1 e η i ] {displaystyle {egin{bmatrix}{dfrac {e^{eta _{1}}}{1+sum _{i=1}^{k-1}e^{eta _{i}}}}\vdots \{dfrac {e^{eta _{k-1}}}{1+sum _{i=1}^{k-1}e^{eta _{i}}}}\{dfrac {1}{1+sum _{i=1}^{k-1}e^{eta _{i}}}}end{bmatrix}}} [ e η 1 ∑ i = 1 k e η i ⋮ e η k ∑ i = 1 k e η i ] {displaystyle {egin{bmatrix}{dfrac {e^{eta _{1}}}{sum _{i=1}^{k}e^{eta _{i}}}}\vdots \{dfrac {e^{eta _{k}}}{sum _{i=1}^{k}e^{eta _{i}}}}end{bmatrix}}} [ e η 1 1 + ∑ i = 1 k − 1 e η i ⋮ e η k − 1 1 + ∑ i = 1 k − 1 e η i 1 1 + ∑ i = 1 k − 1 e η i ] {displaystyle {egin{bmatrix}{dfrac {e^{eta _{1}}}{1+sum _{i=1}^{k-1}e^{eta _{i}}}}\vdots \{dfrac {e^{eta _{k-1}}}{1+sum _{i=1}^{k-1}e^{eta _{i}}}}\{dfrac {1}{1+sum _{i=1}^{k-1}e^{eta _{i}}}}end{bmatrix}}}        + log ⁡ Γ p ( η 2 + p + 1 2 ) = {displaystyle +log Gamma _{p}left(eta _{2}+{frac {p+1}{2}} ight)=} − n 2 log ⁡ | − η 1 | + log ⁡ Γ p ( n 2 ) = {displaystyle -{frac {n}{2}}log |-{oldsymbol {eta }}_{1}|+log Gamma _{p}left({frac {n}{2}} ight)=} ( η 2 + p + 1 2 ) ( p log ⁡ 2 + log ⁡ | V | ) {displaystyle left(eta _{2}+{frac {p+1}{2}} ight)(plog 2+log |mathbf {V} |)}        + log ⁡ Γ p ( η 2 + p + 1 2 ) {displaystyle +log Gamma _{p}left(eta _{2}+{frac {p+1}{2}} ight)}        + log ⁡ Γ p ( − ( η 2 + p + 1 2 ) ) = {displaystyle +log Gamma _{p}left(-{Big (}eta _{2}+{frac {p+1}{2}}{Big )} ight)=} − m 2 log ⁡ | − η 1 | + log ⁡ Γ p ( m 2 ) = {displaystyle -{frac {m}{2}}log |-{oldsymbol {eta }}_{1}|+log Gamma _{p}left({frac {m}{2}} ight)=} − ( η 2 + p + 1 2 ) ( p log ⁡ 2 − log ⁡ | Ψ | ) {displaystyle -left(eta _{2}+{frac {p+1}{2}} ight)(plog 2-log |{oldsymbol {Psi }}|)}        + log ⁡ Γ p ( − ( η 2 + p + 1 2 ) ) {displaystyle +log Gamma _{p}left(-{Big (}eta _{2}+{frac {p+1}{2}}{Big )} ight)}        − ( η 1 + 1 2 ) log ⁡ ( − η 2 + η 3 2 4 η 4 ) {displaystyle -left(eta _{1}+{frac {1}{2}} ight)log left(-eta _{2}+{dfrac {eta _{3}^{2}}{4eta _{4}}} ight)} In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, based on some useful algebraic properties, as well as for generality, as exponential families are in a sense very natural sets of distributions to consider. The term exponential class is sometimes used in place of 'exponential family', or the older term Koopman-Darmois family. The terms 'distribution' and 'family' are often used loosely: properly, an exponential family is a set of distributions, where the specific distribution varies with the parameter; however, a parametric family of distributions is often referred to as 'a distribution' (like 'the normal distribution', meaning 'the family of normal distributions'), and the set of all exponential families is sometimes loosely referred to as 'the' exponential family. The concept of exponential families is credited to E. J. G. Pitman, G. Darmois, and B. O. Koopman in 1935–36. Exponential families of distributions provides a general framework for selecting a possible alternative parameterisation of a parametric family of distributions, in terms of natural parameters, and for defining useful sample statistics, called the natural sufficient statistics of the family. Most of the commonly used distributions form an exponential family or subset of an exponential family, listed in the subsection below. The subsections following it are a sequence of increasingly more general mathematical definitions of an exponential family. A casual reader may wish to restrict attention to the first and simplest definition, which corresponds to a single-parameter family of discrete or continuous probability distributions.