2
$\begingroup$

I have a doubt about the proof of the fact that a positive semi-definite matrix is a covariance matrix. The professor do the following proof:

Let $\Sigma$ be a positive semi-definite $p \times p$ matrix with range $r \leq p$. Then, exists a $p \times r$ matrix $C$ (with range $r$) such that $$\Sigma = C\cdot C^T$$ Then, let $X_1,\ldots,X_r$ be $r$ independent random variables wiht expected value 0 and covariance matrix $I_r$. So, we have that \begin{align*} \text{Cov}(CXC^T) &= C\cdot \text{Cov}(X)\cdot C^T \\ &= C \cdot I_r \cdot C^T \\ &= C \cdot C^T = \Sigma \end{align*} where $X=(X_1,\ldots,X_r)$.

My question is ¿Do the r.v's $X_1,\ldots,X_r$ need to be independent? I think that the only requirement is that $\text{Cov}(X) = I_r$, but my proffesor told me that they HAVE to be independent. I just don't get it.

Thanks for the help.

$\endgroup$
4
  • 1
    $\begingroup$ Are you sure? Why? I know that's true if $X_i$'s are normal distributed but in general it does not need to be true. Take $X$ a Standard Normal r.v and $Y=X^2/\sqrt{2}$. Then $\text{Cov}(Z) = I$ where $Z=(X,Y)$. $\endgroup$ Sep 5, 2019 at 13:24
  • 3
    $\begingroup$ Diagonal covariance matrix means that the variables are not linearly correlated. It does not mean they are independent (i.e. intuitively it does not mean that any function applied to such variables will preserve the 0 correlation). So independence implies a diagonal cov matrix, but the converse does NOT hold true. $\endgroup$
    – Fr1
    Sep 5, 2019 at 13:37
  • $\begingroup$ Here independence is not required. See the good answer below by jld $\endgroup$
    – Fr1
    Sep 5, 2019 at 13:38
  • 1
    $\begingroup$ There is an approach to this that doesn't require independence: you can explicitly write down the characteristic function of a $p$-variate Normal distribution that has $\Sigma$ for its covariance matrix. Unless $\Sigma$ is diagonal, the components of that Normal distribution are not independent. $\endgroup$
    – whuber
    Oct 23, 2019 at 15:52

1 Answer 1

3
$\begingroup$

If I was doing this, I would explicitly use the spectral theorem to say $$ \Sigma = \tilde C \tilde \Lambda\tilde C^T $$ where $\tilde C \in \mathbb R^{p\times p}$ and $\tilde \Lambda$ is a $p\times p$ diagonal matrix with the last $p-r$ entries being exactly zero. Letting $C$ be the first $r$ columns of $C$, and $\Lambda = \text{diag}(\lambda_1, \dots, \lambda_r)$ we then have $$ \Sigma = C\Lambda C^T $$ as the low rank factorization. We can then take $X \sim \mathcal N(\mathbf 0, \Lambda)$ so $$ \text{Var}(CXC^T) = C\Lambda C^T = \Sigma. $$

In this case the elements of $X$ happen to be independent because $\text{Cov}(X_i,X_j) = 0 \iff X_i\perp X_j$ for a multivariate Gaussian, but independence isn't required, just non-correlation.

$\endgroup$
2
  • $\begingroup$ So, the answer to my question is that $X_1,\ldots,X_r$ do not have to be independent?Thanks jld. $\endgroup$ Sep 5, 2019 at 13:37
  • 1
    $\begingroup$ @JuanCorredor that is correct, they just need to have a diagonal covariance matrix which means they're uncorrelated. They certainly could also be independent but it's not required. I chose to use a multivariate Gaussian so in this case they are both uncorrelated and independent but independence wasn't required $\endgroup$
    – jld
    Sep 5, 2019 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.