3
$\begingroup$

On the Wikipedia article for exponential families the density of a distribution on a measure space $(X, \xi)$ from an exponential family is written as $$f_{\theta} \colon X \to \mathbb{R}_{\ge 0}, \qquad x \mapsto h(x) g(\theta) \exp\left( \eta(\theta) \cdot T(x) \right)$$ for $\theta \in \Theta$. On two occasions the Wikipedia articles mentions that "even if $\eta$ is not one-to-one, then ...". In the table of exponential families however, every natural parameter mapping $\eta$ is one-to-one (i.e. bijective onto its image) such that the inverse parameter mapping $\eta^{-1}$ can be written down.

Question 1. Is there an exponential family with non-injective parameter mapping?

Furthermore, it seems that for every exponential family in the above mentioned table, $\eta(\Theta)$ is convex. The aforementioned Wikipedia article states that the natural parameter space is always convex.

Question 2. Does this mean that $\eta(\Theta)$ is always convex? If not, is there an exponential family such that $\eta(\Theta)$ is non-convex?

$\endgroup$
1
  • 1
    $\begingroup$ The general definition means that $\theta$ could essentially be anything, e.g.$$\eta(\theta)=\theta_1\exp(\theta_2^{\theta_3})$$which is not at all useful, but illustrates that the distribution can be overparameterised in $\theta$. $\endgroup$
    – Xi'an
    Jun 21, 2023 at 15:07

1 Answer 1

9
$\begingroup$

Not a complete answer, but primarily a response to

Question 1. Is there an exponential family with non-injective parameter mapping?

The short answer is: yes, there is. It's called the curved exponential family. Now the longer answer.

Let $\mathcal{P} = \{P_\eta:\eta \in \Xi\}$ be a full rank $s$-parameter exponential family with complete sufficient statistic $T$ and consider a sub-model $\mathcal{P}_0$, parametrized by $\theta\in\Theta$, with $\tilde{\eta}(\theta)$ the value of the canonical parameter associated with $\theta$. Thus

$$\mathcal{P}_0 = \{P_{\tilde\eta(\theta)}:\theta\in\Theta\}.$$

Often $\tilde\eta(\theta)$ is a one-to-one mapping from $\Theta$ to $\Xi$ and in such a case, $\mathcal{P} = \mathcal{P}_0$. Curved exponential families may arise when $\mathcal{P}_0$ is a proper subset of $\mathcal{P}$, generally with $\Theta\subseteq\mathbb{R}^r$ and $r<s$. Here are two possibilities:

(a) Points $\eta$ in the range of $\tilde{\eta}$, $\tilde{\eta}(\Omega) = \{\tilde\eta(\theta) : \theta \in\Theta\}$, satisfy a nontrivial linear constraint. In this case, $\mathcal{P}_0$ will be a $q$-parameter exponential family for some $q < s$. The statistic $T$ will still be sufficient, but will not be minimal sufficient.

(b) The points $\eta$ in $\tilde{\eta}(\Omega)$ do not satisfy a linear constraint. In this case, $\mathcal{P}_0$ is called a curved exponential family. Here $T$ will be minimally sufficient, but may not be complete.

A famous example of curved exponential family is $N(\theta,\theta^2)$, $\theta\in\mathbb{R}\setminus\{0\}$. In this case, $$\tilde{\eta}(\theta) = \left(\frac{1}{\theta},-\frac{1}{2\theta^2}\right).$$

The range space $\tilde\eta(\Theta)$ is a parabola (with one point removed). Points in this range space do not satisfy a linear constraint, so in this case, we have a curved exponential family and $T$ is minimal sufficient.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. Does you answer show that for curved exponential families, $\eta(\Theta)$ (in my notation) might not be convex? $\endgroup$ Jun 21, 2023 at 11:59
  • $\begingroup$ @ViktorStein that's an interesting question. In the example of $N(\theta,\theta^2)$ convexity holds, but in general multidimensional cases, my guess is that the parameter space may not always be a convex set. $\endgroup$
    – utobi
    Jul 4, 2023 at 7:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.