Is the covariance matrix of a multivariate normal distribution always invertible?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ No, consider for instance $(X_1,X_2=2X_1)$ when $X_1\sim\mathcal N(0,1).$ $\endgroup$– Xi'anMar 5, 2019 at 13:49
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
If the variables are perfectly correlated, i.e. $\rho=1$, then covariance matrix becomes: $$\Sigma=\begin{bmatrix}\sigma_1^2 & \sigma_1\sigma_2 \\ \sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix}$$
and its determinant is $\Delta=\sigma_1^2\sigma_2^2-\sigma_1\sigma_2\sigma_1\sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=\alpha X_2$ as in @Xian's comment. Here $\alpha>0$, but for $\alpha<0$ $\rho=-1$ which still doesn't save the $\Sigma$.
It is only invertible when $|\rho|<1$ since the covariance matrix is actually $$\Sigma=\begin{bmatrix}\sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix}$$ And, the determinant is $\Delta=\sigma_1^2\sigma_2^2(1-\rho^2)$, which is $>0$ when $|\rho|<1$.
-
$\begingroup$ Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible? $\endgroup$– DadooMar 5, 2019 at 13:55
-
$\begingroup$ Not correlated means $\rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|\rho|<1$, still it is invertible since $\Delta>0$. $\endgroup$– gunesMar 5, 2019 at 13:58
-
7$\begingroup$ This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular. $\endgroup$– whuber ♦Mar 5, 2019 at 14:43
$\begingroup$
$\endgroup$
No.
The covariance matrix of two perfectly correlated standard normal random variables is given by $\Sigma = \pmatrix{1 & 1 \\1 & 1}$, which is not invertible.
