5
$\begingroup$

Consider a vector $r$ of $n$ random variables. Let $\mu = E[r]$ and $\Sigma$ denote the covariance matrix of $r$; that is,

$$\Sigma_{ij} = \sigma_{ij} = E\Bigl[\bigl(r_i - E[r_i]\bigr)\bigl(r_j - E[r_j]\bigr) \Bigr].$$

We know that $\Sigma$ is a positive semidefinite matrix, because we can show that

$$x^T \Sigma x = E\Bigl[ x^T r - E\bigl[x^T r\bigr] \Bigr]^2 \geq 0.$$

(The proof is by writing out the product in summation notation and doing some algebraic manipulation.)

I am currently proofreading a set of practice problems in which the student is asked to consider $r$ having a given $\mu$ and $\Sigma$. Specifically, $n=2$, $\sigma_{1}^2 = 0.1$, $\sigma_{2}^2 = 0.05$, and $\sigma_{12} = \sigma_{21} = -0.4$. Thus, the covariance matrix is

$$\Sigma = \begin{bmatrix} 0.1 & -0.4 \\ -0.4 & 0.05 \end{bmatrix}$$

which is not PSD: If $x = (1, 1)$, then $x^T A x = -0.65 < 0$ as you can verify.

Am I correct to conclude that it is impossible for two random variables to have the given variances and covariances?

(Or are there additional assumptions that went into the proof that $\Sigma$ is PSD that, when relaxed, enable one to construct random variables with the given variances?)

$\endgroup$
2
  • $\begingroup$ Feels like a mistake when specifying the covariance matrix. Just a wild guess. $\endgroup$ Commented Jul 26, 2022 at 2:33
  • $\begingroup$ Because, for instance, the variance $$\pmatrix{1&1}\Sigma\pmatrix{1\\1}=-0.65\lt 0,$$ $\Sigma$ cannot be a covariance matrix. $\endgroup$
    – whuber
    Commented Jul 26, 2022 at 4:54

1 Answer 1

9
$\begingroup$

Covariances cannot have arbitrary values in comparison to variances; $|\operatorname{Cov}(X,Y)| \leq \sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}$. So, Yes, it is not possible to find random variables that have the alleged covariance matrix, which is, as you have discovered, not a positive semidefinite matrix. For the given variances, $|\operatorname{Cov}(X,Y)|$ has maximum value $\frac{\sqrt{2}}{20} \approx 0.0707\cdots$, and the given value $-0.4$ is way out of range,

$\endgroup$
4
  • $\begingroup$ That inequality is just the determinant condition for a $2\times 2$ symmetric matrix to be PSD. Is that where that equality comes from, or does it follow from a prior principle? $\endgroup$
    – Max
    Commented Jul 26, 2022 at 23:05
  • 1
    $\begingroup$ @Max The root of the inequality you are asking about is the Cauchy=-Schwarz Inequality which was known long before covariance matrices (or even matrices) were thought of. $\endgroup$ Commented Jul 27, 2022 at 1:39
  • $\begingroup$ Specifically, Cauchy–Schwartz as applied to the respective definitions of the variance and covariance. $\endgroup$
    – Max
    Commented Jul 27, 2022 at 2:20
  • $\begingroup$ That a covariance matrix must be positive semi-definite can be proved simply by using that a variance cannot be negative. $\endgroup$ Commented Oct 22, 2023 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.