4
$\begingroup$

Is the covariance matrix of a multivariate normal distribution always invertible?

$\endgroup$
  • 1
    $\begingroup$ No, consider for instance $(X_1,X_2=2X_1)$ when $X_1\sim\mathcal N(0,1).$ $\endgroup$ – Xi'an Mar 5 at 13:49
8
$\begingroup$

If the variables are perfectly correlated, i.e. $\rho=1$, then covariance matrix becomes: $$\Sigma=\begin{bmatrix}\sigma_1^2 & \sigma_1\sigma_2 \\ \sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix}$$

and its determinant is $\Delta=\sigma_1^2\sigma_2^2-\sigma_1\sigma_2\sigma_1\sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=\alpha X_2$ as in @Xian's comment. Here $\alpha>0$, but for $\alpha<0$ $\rho=-1$ which still doesn't save the $\Sigma$.

It is only invertible when $|\rho|<1$ since the covariance matrix is actually $$\Sigma=\begin{bmatrix}\sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix}$$ And, the determinant is $\Delta=\sigma_1^2\sigma_2^2(1-\rho^2)$, which is $>0$ when $|\rho|<1$.

$\endgroup$
  • $\begingroup$ Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible? $\endgroup$ – Dadoo Mar 5 at 13:55
  • $\begingroup$ Not correlated means $\rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|\rho|<1$, still it is invertible since $\Delta>0$. $\endgroup$ – gunes Mar 5 at 13:58
  • 7
    $\begingroup$ This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular. $\endgroup$ – whuber Mar 5 at 14:43
6
$\begingroup$

No.

The covariance matrix of two perfectly correlated standard normal random variables is given by $\Sigma = \pmatrix{1 & 1 \\1 & 1}$, which is not invertible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.