I have a doubt about the proof of the fact that a positive semi-definite matrix is a covariance matrix. The professor do the following proof:

Let $\Sigma$ be a positive semi-definite $p \times p$ matrix with range $r \leq p$. Then, exists a $p \times r$ matrix $C$ (with range $r$) such that $$\Sigma = C\cdot C^T$$ Then, let $X_1,\ldots,X_r$ be $r$ independent random variables wiht expected value 0 and covariance matrix $I_r$. So, we have that \begin{align*} \text{Cov}(CXC^T) &= C\cdot \text{Cov}(X)\cdot C^T \\ &= C \cdot I_r \cdot C^T \\ &= C \cdot C^T = \Sigma \end{align*} where $X=(X_1,\ldots,X_r)$.

My question is ¿Do the r.v's $X_1,\ldots,X_r$ need to be independent? I think that the only requirement is that $\text{Cov}(X) = I_r$, but my proffesor told me that they HAVE to be independent. I just don't get it.

Thanks for the help.

  • 1
    $\begingroup$ Are you sure? Why? I know that's true if $X_i$'s are normal distributed but in general it does not need to be true. Take $X$ a Standard Normal r.v and $Y=X^2/\sqrt{2}$. Then $\text{Cov}(Z) = I$ where $Z=(X,Y)$. $\endgroup$ – Juan Corredor Sep 5 '19 at 13:24
  • 3
    $\begingroup$ Diagonal covariance matrix means that the variables are not linearly correlated. It does not mean they are independent (i.e. intuitively it does not mean that any function applied to such variables will preserve the 0 correlation). So independence implies a diagonal cov matrix, but the converse does NOT hold true. $\endgroup$ – Fr1 Sep 5 '19 at 13:37
  • $\begingroup$ Here independence is not required. See the good answer below by jld $\endgroup$ – Fr1 Sep 5 '19 at 13:38
  • 1
    $\begingroup$ There is an approach to this that doesn't require independence: you can explicitly write down the characteristic function of a $p$-variate Normal distribution that has $\Sigma$ for its covariance matrix. Unless $\Sigma$ is diagonal, the components of that Normal distribution are not independent. $\endgroup$ – whuber Oct 23 '19 at 15:52

If I was doing this, I would explicitly use the spectral theorem to say $$ \Sigma = \tilde C \tilde \Lambda\tilde C^T $$ where $\tilde C \in \mathbb R^{p\times p}$ and $\tilde \Lambda$ is a $p\times p$ diagonal matrix with the last $p-r$ entries being exactly zero. Letting $C$ be the first $r$ columns of $C$, and $\Lambda = \text{diag}(\lambda_1, \dots, \lambda_r)$ we then have $$ \Sigma = C\Lambda C^T $$ as the low rank factorization. We can then take $X \sim \mathcal N(\mathbf 0, \Lambda)$ so $$ \text{Var}(CXC^T) = C\Lambda C^T = \Sigma. $$

In this case the elements of $X$ happen to be independent because $\text{Cov}(X_i,X_j) = 0 \iff X_i\perp X_j$ for a multivariate Gaussian, but independence isn't required, just non-correlation.

| cite | improve this answer | |
  • $\begingroup$ So, the answer to my question is that $X_1,\ldots,X_r$ do not have to be independent?Thanks jld. $\endgroup$ – Juan Corredor Sep 5 '19 at 13:37
  • 1
    $\begingroup$ @JuanCorredor that is correct, they just need to have a diagonal covariance matrix which means they're uncorrelated. They certainly could also be independent but it's not required. I chose to use a multivariate Gaussian so in this case they are both uncorrelated and independent but independence wasn't required $\endgroup$ – jld Sep 5 '19 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.