Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

It is well-known that a covariance matrix must be semi-positive definite, however, is the converse true?

That is, does every semi-positive definite matrix correspond to a covariance matrix?

share|improve this question

1 Answer 1

Going by the definitions of PD and PSD here, yes, I think so, since we can do this by construction. I'll assume for a slightly simpler argument that you mean for matrices with real elements, but with appropriate changes it would extend to complex matrices.

Let $A$ be some real PSD matrix; from the definition I linked to, it will be symmetric. Any real symmetric positive definite matrix $A$ can be written as $A = LL^T$. This can be done by $L=Q\sqrt{D}Q^T$ if $A=QDQ^T$ with orthogonal $Q$ and diagonal $D$ and $\sqrt{D}$ as matrix of component wise square roots of $D$. Thus, it needn't be full rank.

Let $Z$ be some vector random variable, of the appropriate dimension, with covariance matrix $I$ (which is easy to create).

Then $LZ$ has covariance matrix $A$.

[At least that's in theory. In practice there'd be various numerical issues to deal with if you wanted good results, and - because of the usual problems with floating point calculation - you'd only approximately get what you need; that is, the population variance of a computed $LZ$ usually wouldn't be exactly $A$. But this sort of thing is always an issue when we come to actually calculate things]

share|improve this answer
    
While it is true that a decomposition $A=LL'$ is possible without full rank, the Cholesky algorithm only works with regular $A$. So without full rank, it cannot be a Cholesky decomposition. Computationally, one could do this decomposition in the singular case by diagonalization. (Although this is far more expensive) –  Horst Grünbusch Aug 13 at 12:14
    
@Horst: Why would $L=Q\sqrt{D}Q^T$ be lower triangular? –  amoeba Aug 13 at 12:41
1  
@amoeba While one could organize it so it is, it doesn't have to be lower triangular for the argument to work - it's a feature of the Cholesky but it's not required for the result to work. –  Glen_b Aug 13 at 13:02
1  
@Glen Is being symmetric a necessary condition for being PSD or is that definition one of many? –  114 Aug 13 at 13:42
1  
@114 for the relation between symmetric and PSD see math.stackexchange.com/questions/516533/… –  Frank Aug 13 at 22:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.