An exponential family distribution in its simplest form is given by

$p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$

where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural parameter and $A(\theta)$ is the log normalizer.

Assume that $x\in \mathbb{R}$. I would like to have $p(x|\theta)$ that

  1. Has $T(x)$ with length of at least 3.
  2. Can be easily sampled. Perhaps with inverse transform sampling or other ways.
  3. $A(\theta)$ can be computed easily.

Many well-known distributions in the family such as Gaussian, Gamma, Beta do not satisfy (1). What is a non-trivial example of such $p(x|\theta)$ ? Having, for example, $T(x) = (1, 2, 3, x, x^2)^\top$ is not what I am looking for. Thanks !

  • $\begingroup$ There are some examples on the Wikipedia page for exponential family: en.wikipedia.org/wiki/Exponential_family $\endgroup$
    – jbowman
    Jan 8 '15 at 17:53
  • $\begingroup$ Did you try integrating using $(x,x^2,x^3)$? $\endgroup$
    – Neil G
    Jan 8 '15 at 18:39
  • $\begingroup$ @NeilG I thought about that. But not sure how to sample from it. $\endgroup$
    – wij
    Jan 9 '15 at 8:33
  • $\begingroup$ If you can integrate, just use the inverse CDF. $\endgroup$
    – Neil G
    Jan 9 '15 at 8:34

If you take for $T$ the vector of indicators, $$T(x)=(\mathbb{I}_{[0,1]}(x),\mathbb{I}_{(1,2]}(x),\mathbb{I}_{(2,3]}(x))$$ and assume that the dominating measure is the uniform distribution on $[0,4]$, you have a solution to your problem as

  1. $T$ has length $3$ and is sufficient;
  2. The density $p(x|\theta)$ is a step function $$p(x|\theta)\propto\begin{cases} e^{\theta_1} &\text{if } 0\le x\le1\\ e^{\theta_2} &\text{if } 1<x\le2\\ e^{\theta_3} &\text{if } 2< x\le 3\\ 1 &\text{if } 3<x\le4\\\end{cases}$$ hence easy to sample;
  3. The normalisation is equally straightforward$$A(\theta)=\log(e^{\theta_1} +e^{\theta_2}+e^{\theta_3}+1)$$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.