Doubt regarding the variance and covariance matrix

Question

I have a general doubt regarding the variance and covariance matrix. I know that the general structure of variance and covariance matrix is such that it's diagonal elements represents the variance and non diagonal elements represents the covariance. Also, for a $2×2$ matrix, the non diagonal elements will represent the same covariance term. The diagonal terms cannot be negative as they represents variance term. Then, my doubt is why the following cannot be a variance covariance matrix, since it satisfies all the properties which is applicable for such matrices:

$\begin{pmatrix} 1 &2 \\ 2 & 2 \end{pmatrix}$

Thanks.

Answer 1

A real matrix is a covariance matrix if and only if it is symmetric positive semi-definite. A consequence of this is that all diagonal elements must be non-negative. So diagonal elements being non-negative is necessary for a matrix to be a covariance matrix, but it is not sufficient.

Indeed, to be a covariance matrix, the matrix must be symmetric, and have all eigenvalues non-negative. The eigenvalues are the variances in the independent coordinate frame (i.e., after the matrix has been rotated to be diagonal). So for a real symmetric matrix to be a covariance matrix, it is necessary for all of its eigenvalues to be non-negative (and it does not suffice for all diagonal elements, i.e., variances, to be non-negative, unless the matrix is diagonal, in which case the variances, i.e., the diagonal elements, are the eigenvalues).

Answer 2

If $\mathrm{Cov}(X,Y)=2$, $\mathrm{Var}(X)=1$, $\mathrm{Var}(Y)=2$ then $$E(XY)-E(X)E(Y)=2$$ $$E(X^2)-E(X)^2=1$$ $$E(Y^2)-E(Y)^2=2$$ Now the means can be anything without affecting the variance, so let' assume $E(X)=E(Y)=1$. Then we get $E(XY)=3$, $E(X^2)=2$, $E(Y^2)=3$. But then $E((X-Y)^2)=2-2\cdot 3+3<0$ which is impossible.

---

If you're wondering where that came from, it was inspired by a proof of the Cauchy-Schwartz inequality. (In general the key phrase is that covariance matrices must be positive semidefinite.) Note $$0\le E[(X+aY)^2]=1+\mu_X^2+2a(2+\mu_X\mu_Y)+a^2(2+\mu_Y^2)=(\mu_X+a\mu_Y)^2+1+4a+2a^2$$ (where $\mu_X=E(X)$ and $\mu_Y=E(Y)$). But if $a=-\mu_X/\mu_Y$ and $\mu_Y=1=\mu_X$, then this is impossible.