2
$\begingroup$

I have a few questions with respect to the gaussian distribution, its moments and independence.

So a gaussian distribution is fully specified by its first two moments, the mean and variance (or covariance matrix in the multivariate case). Since uncorrelatedness means unrelated in the first two statistical moments and independence means unrelatedness in all higher moments as well, the jointly gaussian distributed variables are independent if they re uncorrelated.

My question now is, what does it mean that it is fully specified by its first two moments? I mean the higher order moments are not zero? Can someone maybe also give a counter example, so how it looks if a distribution is not fully specified by its first two moments?

And another questions relates to the rotational invariance of the gaussian. For independent component analysis one demands that the latent variables are not gaussian distributed because the gaussian can -apparently - only be defined up to a rotation. What exactly does that mean and does it have anything to do with its property that it is fully specified by its first two moments?

Improve this question
$\endgroup$
4
  • $\begingroup$ Given the first two moments, one can write down the density of the distribution. $\endgroup$
    – Xi'an
    Dec 14, 2018 at 8:42
  • $\begingroup$ but doesnt this apply to almost all distributions? I mean the first moment is the mean of the distribution and the second moment is the variance+mean². And e.g. the Bernoulli distribution has mean $\pi$ which is also its only parameter and can therefore also be directly written down by knowing even only the first moment $\endgroup$
    – guest1
    Dec 14, 2018 at 9:32
  • $\begingroup$ This is true of many distributions and hence not a property unique to the Gaussian distribution. $\endgroup$
    – Xi'an
    Dec 14, 2018 at 9:47
  • $\begingroup$ Statistical terminology in these matters is awful, because many writers use the word "distribution" to refer either to (a) the distribution function of a random variable or (b) a family of such distributions. Your questions move about between the two senses, potentially creating much confusion. In particular, "fully specified by its first two moments" is a concept that can only apply to a family and it merely means there is a one-to-one correspondence between the first two moments and a parameter for the family. See stats.stackexchange.com/questions/320746 for more. $\endgroup$
    – whuber ♦
    Dec 14, 2018 at 15:03

1 Answer 1

Reset to default
2
$\begingroup$

Distributions that involve more than two parameters are rarely characterised by their first two moments. An example is the g-and-k distribution, which quantile function $$q_{A,B,c,g,k}(z) = A + B [1 + c \tanh(gz/2)] z \exp(k z^2/2)$$ involves five parameters. Among other characteristics, it does not enjoy a closed-form cdf or pdf (but they can be numerically approximated). Another extreme example is when considering the set of all probability distributions on $\mathbb{N}$, which are indexed by an infinite sequence $(p_n)_{n\in\mathbb{N}}$.

The rotational invariance of the Normal distribution is restricted to the case of an iid vector with mean zero, $X\sim\mathcal{N}_d(0_d,I_d)$ since $RX\sim \mathcal{N}_d(0_d,R R^\text{T})$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you. How does this property, that the gaussian is fully defined by its first two moments, lead to the fact that if jointly gaussian variables are uncorrelated they are also statstically independent? So would this apply also for all other distributions that are characterized by their first two moments? $\endgroup$
    – guest1
    Dec 14, 2018 at 10:03
  • $\begingroup$ Sorry, what do you mean?^^ $\endgroup$
    – guest1
    Dec 14, 2018 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.