# Is there an exponential family such that its natural parameter mapping is non-invertible or has non-convex range?

On the Wikipedia article for exponential families the density of a distribution on a measure space $$(X, \xi)$$ from an exponential family is written as $$f_{\theta} \colon X \to \mathbb{R}_{\ge 0}, \qquad x \mapsto h(x) g(\theta) \exp\left( \eta(\theta) \cdot T(x) \right)$$ for $$\theta \in \Theta$$. On two occasions the Wikipedia articles mentions that "even if $$\eta$$ is not one-to-one, then ...". In the table of exponential families however, every natural parameter mapping $$\eta$$ is one-to-one (i.e. bijective onto its image) such that the inverse parameter mapping $$\eta^{-1}$$ can be written down.

Question 1. Is there an exponential family with non-injective parameter mapping?

Furthermore, it seems that for every exponential family in the above mentioned table, $$\eta(\Theta)$$ is convex. The aforementioned Wikipedia article states that the natural parameter space is always convex.

Question 2. Does this mean that $$\eta(\Theta)$$ is always convex? If not, is there an exponential family such that $$\eta(\Theta)$$ is non-convex?

• The general definition means that $\theta$ could essentially be anything, e.g.$$\eta(\theta)=\theta_1\exp(\theta_2^{\theta_3})$$which is not at all useful, but illustrates that the distribution can be overparameterised in $\theta$. Jun 21 at 15:07

Not a complete answer, but primarily a response to

Question 1. Is there an exponential family with non-injective parameter mapping?

The short answer is: yes, there is. It's called the curved exponential family. Now the longer answer.

Let $$\mathcal{P} = \{P_\eta:\eta \in \Xi\}$$ be a full rank $$s$$-parameter exponential family with complete sufficient statistic $$T$$ and consider a sub-model $$\mathcal{P}_0$$, parametrized by $$\theta\in\Theta$$, with $$\tilde{\eta}(\theta)$$ the value of the canonical parameter associated with $$\theta$$. Thus

$$\mathcal{P}_0 = \{P_{\tilde\eta(\theta)}:\theta\in\Theta\}.$$

Often $$\tilde\eta(\theta)$$ is a one-to-one mapping from $$\Theta$$ to $$\Xi$$ and in such a case, $$\mathcal{P} = \mathcal{P}_0$$. Curved exponential families may arise when $$\mathcal{P}_0$$ is a proper subset of $$\mathcal{P}$$, generally with $$\Theta\subseteq\mathbb{R}^r$$ and $$r. Here are two possibilities:

(a) Points $$\eta$$ in the range of $$\tilde{\eta}$$, $$\tilde{\eta}(\Omega) = \{\tilde\eta(\theta) : \theta \in\Theta\}$$, satisfy a nontrivial linear constraint. In this case, $$\mathcal{P}_0$$ will be a $$q$$-parameter exponential family for some $$q < s$$. The statistic $$T$$ will still be sufficient, but will not be minimal sufficient.

(b) The points $$\eta$$ in $$\tilde{\eta}(\Omega)$$ do not satisfy a linear constraint. In this case, $$\mathcal{P}_0$$ is called a curved exponential family. Here $$T$$ will be minimally sufficient, but may not be complete.

A famous example of curved exponential family is $$N(\theta,\theta^2)$$, $$\theta\in\mathbb{R}\setminus\{0\}$$. In this case, $$\tilde{\eta}(\theta) = \left(\frac{1}{\theta},-\frac{1}{2\theta^2}\right).$$

The range space $$\tilde\eta(\Theta)$$ is a parabola (with one point removed). Points in this range space do not satisfy a linear constraint, so in this case, we have a curved exponential family and $$T$$ is minimal sufficient.

• Thank you for your answer. Does you answer show that for curved exponential families, $\eta(\Theta)$ (in my notation) might not be convex? Jun 21 at 11:59
• @ViktorStein that's an interesting question. In the example of $N(\theta,\theta^2)$ convexity holds, but in general multidimensional cases, my guess is that the parameter space may not always be a convex set. Jul 4 at 7:39