# What the dimension of an exponential family tell us about that family?

In Wikipedia it is stated that:

A vector exponential family is said to be curved if the dimension of $${\boldsymbol \theta} = \left (\theta_1, \theta_2, \ldots, \theta_d \right )^T$$ is less than the dimension of the vector $${\boldsymbol \eta}(\boldsymbol \theta) = \left (\eta_1(\boldsymbol \theta), \eta_2(\boldsymbol \theta), \ldots, \eta_s(\boldsymbol \theta) \right )^T.$$

with

$$f_X(x|\boldsymbol \theta) = h(x) g(\boldsymbol \theta) \exp\Big(\boldsymbol\eta({\boldsymbol \theta}) \cdot \mathbf{T}(x)\Big)$$

but what happens when the dimension of $\boldsymbol \eta$ is smaller than the one of $\boldsymbol \theta$ ?

The dimension of the parameter vector $\theta$ can't be larger than that of the natural parameters $\eta$ because then you have necessarily redundant $\theta$. The idea of a curved exponential family is that you're slicing out part of the natural parameter space. If you have extra parameters then you are still taking the whole natural parameter space, and it's just a regular exponential family.
• Thanks. From "The Bayesian choice", C. Robert p 138: $x \sim N(\theta, \theta^2)$ induces an exponential family of dimension 2. Is not it contradictory with our answer ? Thanks in advance. Jun 12, 2014 at 12:42
• @peuhp There seem to be two senses of the word "dimension" in operation here: one is the dimension of a manifold (coordinatized by the parameters $\theta$, of dimension $d$) and the other is a dimension of a space in which it is embedded (with coordinates $\eta$, of dimension $s$). The latter was likely meant in the C. Robert quotation--but whether that is the correct interpretation depends on exactly how Robert defines "dimension."
• @whuber. Thanks. From the book (and roughly speaking), the dimension of the exponential family is the one of $\nu(\theta)$, so I am a bit lost. It seems to me still contradictory: $\theta$ is a scalar ($d=1$) but $\nu(\theta)$ is cleary a 2d vector ($s=2$). So $d>s$. Thanks for helping me to clarify my mind. Jun 12, 2014 at 15:13
• @peuhp: Isn't your example exactly the one defined to be a curved exponential family? The dimension of $\theta$ is 1; the dimension of $\eta$ is 2. It's a parabolic slice through the cartesian plane. Jun 12, 2014 at 15:24