An exponential family distribution in its simplest form is given by

$p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$

where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural parameter and $A(\theta)$ is the log normalizer.

Assume that $x\in \mathbb{R}$. I would like to have $p(x|\theta)$ that

  1. Has $T(x)$ with length of at least 3.
  2. Can be easily sampled. Perhaps with inverse transform sampling or other ways.
  3. $A(\theta)$ can be computed easily.

Many well-known distributions in the family such as Gaussian, Gamma, Beta do not satisfy (1). What is a non-trivial example of such $p(x|\theta)$ ? Having, for example, $T(x) = (1, 2, 3, x, x^2)^\top$ is not what I am looking for. Thanks !

  • $\begingroup$ There are some examples on the Wikipedia page for exponential family: en.wikipedia.org/wiki/Exponential_family $\endgroup$
    – jbowman
    Jan 8, 2015 at 17:53
  • $\begingroup$ Did you try integrating using $(x,x^2,x^3)$? $\endgroup$
    – Neil G
    Jan 8, 2015 at 18:39
  • $\begingroup$ @NeilG I thought about that. But not sure how to sample from it. $\endgroup$
    – wij
    Jan 9, 2015 at 8:33
  • $\begingroup$ If you can integrate, just use the inverse CDF. $\endgroup$
    – Neil G
    Jan 9, 2015 at 8:34

1 Answer 1


If you take for $T$ the vector of indicators, $$T(x)=(\mathbb{I}_{[0,1]}(x),\mathbb{I}_{(1,2]}(x),\mathbb{I}_{(2,3]}(x))$$ and assume that the dominating measure is the uniform distribution on $[0,4]$, you have a solution to your problem as

  1. $T$ has length $3$ and is sufficient;
  2. The density $p(x|\theta)$ is a step function $$p(x|\theta)\propto\begin{cases} e^{\theta_1} &\text{if } 0\le x\le1\\ e^{\theta_2} &\text{if } 1<x\le2\\ e^{\theta_3} &\text{if } 2< x\le 3\\ 1 &\text{if } 3<x\le4\\\end{cases}$$ hence easy to sample;
  3. The normalisation is equally straightforward$$A(\theta)=\log(e^{\theta_1} +e^{\theta_2}+e^{\theta_3}+1)$$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.