# Exponential family distribution with high-order statistics

An exponential family distribution in its simplest form is given by

$p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$

where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural parameter and $A(\theta)$ is the log normalizer.

Assume that $x\in \mathbb{R}$. I would like to have $p(x|\theta)$ that

1. Has $T(x)$ with length of at least 3.
2. Can be easily sampled. Perhaps with inverse transform sampling or other ways.
3. $A(\theta)$ can be computed easily.

Many well-known distributions in the family such as Gaussian, Gamma, Beta do not satisfy (1). What is a non-trivial example of such $p(x|\theta)$ ? Having, for example, $T(x) = (1, 2, 3, x, x^2)^\top$ is not what I am looking for. Thanks !

• There are some examples on the Wikipedia page for exponential family: en.wikipedia.org/wiki/Exponential_family Jan 8 '15 at 17:53
• Did you try integrating using $(x,x^2,x^3)$? Jan 8 '15 at 18:39
• @NeilG I thought about that. But not sure how to sample from it.
– wij
Jan 9 '15 at 8:33
• If you can integrate, just use the inverse CDF. Jan 9 '15 at 8:34

If you take for $T$ the vector of indicators, $$T(x)=(\mathbb{I}_{[0,1]}(x),\mathbb{I}_{(1,2]}(x),\mathbb{I}_{(2,3]}(x))$$ and assume that the dominating measure is the uniform distribution on $[0,4]$, you have a solution to your problem as
1. $T$ has length $3$ and is sufficient;
2. The density $p(x|\theta)$ is a step function $$p(x|\theta)\propto\begin{cases} e^{\theta_1} &\text{if } 0\le x\le1\\ e^{\theta_2} &\text{if } 1<x\le2\\ e^{\theta_3} &\text{if } 2< x\le 3\\ 1 &\text{if } 3<x\le4\\\end{cases}$$ hence easy to sample;
3. The normalisation is equally straightforward$$A(\theta)=\log(e^{\theta_1} +e^{\theta_2}+e^{\theta_3}+1)$$