This might be a very naive question.

Wikipedia describes an exponential family as a distribution

$$f(x \mid \theta) = h(x) \exp( - \theta x - A(\theta)),$$

where $$A(\theta) = \log\left(\int h(x) \exp( - \theta x) dx\right).$$

I want to understand the role of the underlying measure or basically the $h(x)$ in the above equation.

For instance, can any function of $\theta$ be written as $A(\theta)$ for some choice of $h(x)$.

  • 3
    $\begingroup$ $A$ is the Laplace transform of $h$. Depending on what you mean by "any function," the Laplace transform can be inverted. $\endgroup$
    – whuber
    Oct 20 '15 at 16:31
  • $\begingroup$ $A(x)$ would actually be $\log\left(\int h(x) \exp(-\theta x)\right)$ $\endgroup$ Dec 20 '17 at 17:19

This is neither a complete nor thorough answer, but an illustration instead.

Consider the product space $\{0, 1\}^n$ that serves as sample space for the extraction of $n$ i.i.d. Bernoulli r.v.s with parameter $p\in(0, 1)$ $$ P(X_1=x_1, \ldots, X_n=x_n) = p^{x_1+\ldots+x_n}(1-p)^{n-x_1-\ldots-x_n} $$ and their transformation into the binomial r.v. $X:=X_1+\ldots+X_n \sim\text{Binom}(n, p)$ with $$ P(X=x) = \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}, $$ which can be written as member of the exponential family in the form $$ P(X=x) = \exp\left\{ x \cdot \log\left(\frac{p}{1-p}\right) + n \log(1-p) + \log\left(\frac{n!}{x!(n-x)!}\right) \right\}, $$ which would give $A(p)=n \log(1-p)$ and $h(x)=\log\left(\frac{n!}{x!(n-x)!}\right)$.

In that example, $h(x)$ accounts for the multiplicity of elements $(x_1, \ldots, x_n)$ of $\{0, 1\}^n$ which yield $x_1+\ldots+x_n=x$ $$ h(x) = \log\left( \#\{ (x_1, \ldots, x_n)\in\{0, 1\}^n : x_1+\ldots+x_n=x \} \right) = \log\left(\frac{n!}{x!(n-x)!}\right). $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.