I have some covariance matrix $$A = \begin{bmatrix}121 & c\\c & 81\end{bmatrix} $$

The problem is to determine the possible values of $c$.

Now I know that the elements of this matrix are given by the usual definition of the covariance,

$$ \frac{1}{N-1} \sum_{i=1}^N (X_i - \bar{x})(Y_i - \bar{y})$$

and so e.g.

$$ \frac{1}{N-1} \sum_{i=1}^N (X_i - \bar{x})^2 = 121$$

$$ \frac{1}{N-1} \sum_{i=1}^N (Y_i - \bar{y})^2 = 81$$

But I can't see how to go from here to determining $c$?

  • 2
    $\begingroup$ is it always 2x2 matrix? $\endgroup$ – Aksakal Apr 16 at 13:58

You might find it instructive to start with a basic idea: the variance of any random variable cannot be negative. (This is clear, since the variance is the expectation of the square of something and squares cannot be negative.)

Any $2\times 2$ covariance matrix $\mathbb A$ explicitly presents the variances and covariances of a pair of random variables $(X,Y),$ but it also tells you how to find the variance of any linear combination of those variables. This is because whenever $a$ and $b$ are numbers,

$$\operatorname{Var}(aX+bY) = a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) + 2ab\operatorname{Cov}(X,Y) = \pmatrix{a&b}\mathbb A\pmatrix{a\\b}.$$

Applying this to your problem we may compute

$$\begin{aligned} 0 \le \operatorname{Var}(aX+bY) &= \pmatrix{a&b}\pmatrix{121&c\\c&81}\pmatrix{a\\b}\\ &= 121 a^2 + 81 b^2 + 2c^2 ab\\ &=(11a)^2+(9b)^2+\frac{2c}{(11)(9)}(11a)(9b)\\ &= \alpha^2 + \beta^2 + \frac{2c}{(11)(9)} \alpha\beta. \end{aligned}$$

The last few steps in which $\alpha=11a$ and $\beta=9b$ were introduced weren't necessary, but they help to simplify the algebra. In particular, what we need to do next (in order to find bounds for $c$) is complete the square: this is the process emulating the derivation of the quadratic formula to which everyone is introduced in grade school. Writing

$$C = \frac{c}{(11)(9)},\tag{*}$$

we find

$$\alpha^2 + \beta^2 + \frac{2c^2}{(11)(9)} \alpha\beta = \alpha^2 + 2C\alpha\beta + \beta^2 = (\alpha+C\beta)^2+(1-C^2)\beta^2.$$

Because $(\alpha+C\beta)^2$ and $\beta^2$ are both squares, they are not negative. Therefore if $1-C^2$ also is non-negative, the entire right side is not negative and can be a valid variance. Conversely, if $1-C^2$ is negative, you could set $\alpha=-c\beta$ to obtain the value $(1-C^2)\beta^2\lt 0$ on the right hand side, which is invalid.

You therefore deduce (from these perfectly elementary algebraic considerations) that

If $A$ is a valid covariance matrix, then $1-C^2$ cannot be negative.

Equivalently, $|C|\le 1,$ which by $(*)$ means $-(11)(9) \le c \le (11)(9).$

There remains the question whether any such $c$ does correspond to an actual variance matrix. One way to show this is true is to find a random variable $(X,Y)$ with $\mathbb A$ as its covariance matrix. Here is one way (out of many).

I take it as given that you can construct independent random variables $A$ and $B$ having unit variances: that is, $\operatorname{Var}(A)=\operatorname{Var}(B) = 1.$ (For example, let $(A,B)$ take on the four values $(\pm 1, \pm 1)$ with equal probabilities of $1/4$ each.)

The independence implies $\operatorname{Cov}(A,B)=0.$ Given a number $c$ in the range $-(11)(9)$ to $(11)(9),$ define random variables

$$X = \sqrt{11^2-c^2/9^2}A + (c/9)B,\quad Y = 9B$$

(which is possible because $11^2 - c^2/9^2\ge 0$) and compute that the covariance matrix of $(X,Y)$ is precisely $\mathbb A.$

Finally, if you carry out the same analysis for any symmetric matrix $$\mathbb A = \pmatrix{a & b \\ b & d},$$ you will conclude three things:

  1. $a \ge 0.$

  2. $d \ge 0.$

  3. $ad - b^2 \ge 0.$

These conditions characterize symmetric, positive semi-definite matrices. Any $2\times 2$ matrix satisfying these conditions indeed is a variance matrix. (Emulate the preceding construction.)

  • 4
    $\begingroup$ It might be worth mentioning that $C$ here is the correlation and, as shown, is always between $-1$ and $+1$ $\endgroup$ – Henry Apr 17 at 12:00

An intuitive method to determine this answer quickly is to just remember that covariance matrices may be interpreted in the form

\begin{equation} A = \begin{pmatrix} \sigma_1^2 & \rho_{12}\sigma_1\sigma_2 &\rho_{13}\sigma_1\sigma_3 & \cdots & \rho_{1n}\sigma_1 \sigma_n \\ & \sigma_2^2 & \rho_{23}\sigma_2\sigma_3 & \cdots & \rho_{2n}\sigma_2 \sigma_n \\ & & \sigma_3^2 & \cdots & \rho_{3n}\sigma_3 \sigma_n \\ & & & \ddots & \vdots \\ & & & & \sigma_n^2 \end{pmatrix} \end{equation}

where $\rho_{ab} \in [-1,1]$ is a Pearson Correlation Coefficient. In your case you have

\begin{align} \sigma_1^2 = 121 ,~~~ \sigma_2^2 = 81 ~\Longrightarrow ~ |c| \leq \sqrt{121\cdot 81} = 99 \end{align}

i.e. $c \in [-99, 99]$.

  • 4
    $\begingroup$ +1 This is a great answer for readers familiar with this representation of covariance matrices. I don't think it would be remiss, though, to point out that when $n\gt 2$ the correlation coefficients are subject to additional restrictions: it doesn't suffice for them all just to lie between $-1$ and $1.$ The case $n=3$ is discussed at stats.stackexchange.com/questions/72790. $\endgroup$ – whuber Apr 17 at 18:05
  • 1
    $\begingroup$ @whuber This is a good point. Just to clarify, it was my intention to imply that given only the diagonal of the covariance matrix, then the feasible set for an individual correlation is $[-1,1]$. However, you are correct in pointing out that the feasible set may be smaller when given additional information (e.g. if provided some off-diagonal entries). $\endgroup$ – jodag Apr 17 at 18:38

$A$ is posdef so by Sylvesters criterion $det(A) = 121 \cdot 81 - c^2 \geq 0$. Thus, any $c \in [-99, 99]$ will produce a valid covariance matrix.

  • $\begingroup$ Covariance matrices can be positive semi-definite right? Does the semi here change anything? $\endgroup$ – user1887919 Apr 16 at 15:23
  • 2
    $\begingroup$ semi is the case when c is exactly 99 and det = 0. $\endgroup$ – Hunaphu Apr 16 at 16:16
  • $\begingroup$ Constructive comments about your voting are always welcome. $\endgroup$ – whuber Apr 16 at 17:14
  • 6
    $\begingroup$ "posdef" might be a bit misleading here ... I think it's worth writing "positive semi-definite" as @user1887919 says $\endgroup$ – Silverfish Apr 16 at 23:39
  • 2
    $\begingroup$ @Hunaphu $c=+99$ and $c=-99$ both lead to zero determinant. If $-99 < c < +99$ then you would have a strictly positive definite matrix $\endgroup$ – Henry Apr 17 at 12:03

There are three main possibilities of note. One is that the variable are uncorrelated, in which case the off-diagonal entries are easy to calculate as 0. Another possibility is that you don't really have two different variables. $y$ is simply a scalar multiple of $x$ (i.e. perfect correlation). If $y= c x$, then $\sigma_{xy} =\sigma_{x}\sigma_{xy}=99$. We get a third possibility in noting that the above assumes $c>0$. For $c<0$, we get $\sigma_{xy} =-99$.

Geometrically, the covariance between two vectors is the product of their lengths times the cosine of the angle between them. Since the cosine varies from $-1$ to $1$, the covariance ranges from the product of their lengths to the negative of the product.

Another approach is to consider $z_1 = \frac{x-\mu_{x}}{\sigma_{x}}$ and $z_2 = \frac{y-\mu_y}{\sigma_{y}}$. $\sigma_{xy} = \sigma_{(\sigma_x z_1)(\sigma_y z_2)}=\sigma_x \sigma_y \sigma_{z_1z_2}=99\sigma_{z_1z_2}$ and $\sigma_{z_1z_2}$ is simply the correlation between $x$ and $y$, which ranges from $-1$ to $1$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.