9
$\begingroup$

Suppose a multivariate distribution over $\mathbb R^n$ has a singular covariance matrix. Can we conclude that it does not have a density function?

For example, it is the case for the multivariate normal distribution, but I am not sure if it is true for all other multivariate distributions.

This is, I think, a question of the existence of Radon-Nikodym derivative wrt the Lebesgue measure on $\mathbb R^n$ , but elementary probability theory may also have the answer.

$\endgroup$

3 Answers 3

7
$\begingroup$

A singular covariance matrix means that there exists a linear combination $Y = \sum_{i=1}^n a_i X_i$ of the $n$ random variables such that $E[Y] = a_0$ and $\operatorname{var}(Y) = 0$. Thus, all the probability mass lies in a hyperplane of $\mathbb R^n$ defined by $\sum_{i=1}^n a_i x_i = a_0$ and so the $n$ random variables cannot have a $n$-variate density function.

$\endgroup$
0
3
$\begingroup$

Yes, but it will be a probability distribution over a lower dimensional subspace. You could argue that it is a probability distribution in R^N if you allow things like dirac delta functions. That's a subtle mathematical issue but physicists, for example, do it all the time.

$\endgroup$
2
$\begingroup$

Although this is alluded to above, I want to make it clearer that whilst it may not have a meaningful density on $\mathbb{R}^{n}$ you can define the density on a Rank($\Sigma$)-dimensional subspace, where $\Sigma$ denotes the covariance matrix.

$\endgroup$
1
  • 2
    $\begingroup$ This is not generally true. Take, for instance, $(X,Y)$ for which $Y=0$ and $X$ has any nondegenerate non-continuous distribution. Although the rank of the covariance matrix is $1$, this distribution does not have a density on any one-dimensional submanifold of $\mathbb{R}^2$. $\endgroup$
    – whuber
    Commented Jun 3, 2014 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.