I want to understand how the variance of an exponential family behaves. To take a very concrete example. Let consider the unit ball $B$ in d dimensions.

Consider the following distribution over unit ball (parametrized by $\theta$) $$P_{\theta}(x) = \frac{exp(-\theta^T x)}{Z(\theta)}$$

where $Z(\theta)$ is the normalizing constant.

$P_0$ is the uniform distribution

Are the following statements true?

$\forall \theta \;\;$ Variance of $P_0$ $\geq$ Variance of $P_{\theta}$

More generally

$\forall \theta_1, \theta_2 \|\theta_1\| > \|\theta_2\| \;\;$ Variance of $P_{\theta_1}$ $\leq$ Variance of $P_{\theta_2}$

  • $\begingroup$ By variance here do you mean covariance matrix, as $x$ is d-dimensional? If so, how do you define $\leq$ for covariance matrices? $\endgroup$ Apr 19 '16 at 14:06
  • $\begingroup$ What about a beta distribution with $\alpha,\beta\leq1$, which is from an exponential family and has more variance than the uniform? $\endgroup$ Apr 19 '16 at 14:41
  • $\begingroup$ @JuhoKokkala - My bad. I was a bit sloppy with my notations. To start with I am fine with getting this for d = 1 where there is no confusion. In higher dimensions I would replace $\leq$ in the Positive Semi Definite sense applied to the covariance matrix $\endgroup$ Apr 19 '16 at 15:01
  • $\begingroup$ @ChristophHanck - I am not totally sure if this answers my question. To my understanding the Beta distribution needs a reparametrization to be expressed as an exponential family distribution. In that case I dont think the precise form of my question holds. $\endgroup$ Apr 19 '16 at 15:03
  • 1
    $\begingroup$ If it's a member of the exponential family, it's a member of the exponential family regardless of parameterization. It may be that the standard way of writing the distribution doesn't use the "natural" parameterization, where "natural" refers to the exponential family-related definition of same, but the distribution itself is still a member of the exponential family. $\endgroup$
    – jbowman
    Apr 19 '16 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.