Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Suppose a multivariate distribution over $\mathbb R^n$ has a singular covariance matrix. Can we conclude that it does not have a density function?

For example, it is the case for the multivariate normal distribution, but I am not sure if it is true for all other multivariate distributions.

This is, I think, a question of the existence of Radon-Nikodym derivative wrt the Lebesgue measure on $\mathbb R^n$ , but elementary probability theory may also have the answer.

share|improve this question

3 Answers 3

A singular covariance matrix means that there exists a linear combination $Y = \sum_{i=1}^n a_i X_i$ of the $n$ random variables such that $E[Y] = a_0$ and $\operatorname{var}(Y) = 0$. Thus, all the probability mass lies in a hyperplane of $\mathbb R^n$ defined by $\sum_{i=1}^n a_i x_i = a_0$ and so the $n$ random variables cannot have a $n$-variate density function.

share|improve this answer

Although this is alluded to above, I want to make it clearer that whilst it may not have a meaningful density on $\mathbb{R}^{n}$ you can define the density on a Rank($\Sigma$)-dimensional subspace, where $\Sigma$ denotes the covariance matrix.

share|improve this answer
    
This is not generally true. Take, for instance, $(X,Y)$ for which $Y=0$ and $X$ has any nondegenerate non-continuous distribution. Although the rank of the covariance matrix is $1$, this distribution does not have a density on any one-dimensional submanifold of $\mathbb{R}^2$. –  whuber Jun 3 at 20:22

Yes, but it will be a probability distribution over a lower dimensional subspace. You could argue that it is a probability distribution in R^N if you allow things like dirac delta functions. That's a subtle mathematical issue but physicists, for example, do it all the time.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.