# Determine the off - diagonal elements of covariance matrix, given the diagonal elements

I have some covariance matrix $$A = \begin{bmatrix}121 & c\\c & 81\end{bmatrix}$$

The problem is to determine the possible values of $$c$$.

Now I know that the elements of this matrix are given by the usual definition of the covariance,

$$\frac{1}{N-1} \sum_{i=1}^N (X_i - \bar{x})(Y_i - \bar{y})$$

and so e.g.

$$\frac{1}{N-1} \sum_{i=1}^N (X_i - \bar{x})^2 = 121$$

$$\frac{1}{N-1} \sum_{i=1}^N (Y_i - \bar{y})^2 = 81$$

But I can't see how to go from here to determining $$c$$?

• is it always 2x2 matrix? Apr 16 at 13:58

You might find it instructive to start with a basic idea: the variance of any random variable cannot be negative. (This is clear, since the variance is the expectation of the square of something and squares cannot be negative.)

Any $$2\times 2$$ covariance matrix $$\mathbb A$$ explicitly presents the variances and covariances of a pair of random variables $$(X,Y),$$ but it also tells you how to find the variance of any linear combination of those variables. This is because whenever $$a$$ and $$b$$ are numbers,

$$\operatorname{Var}(aX+bY) = a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) + 2ab\operatorname{Cov}(X,Y) = \pmatrix{a&b}\mathbb A\pmatrix{a\\b}.$$

Applying this to your problem we may compute

\begin{aligned} 0 \le \operatorname{Var}(aX+bY) &= \pmatrix{a&b}\pmatrix{121&c\\c&81}\pmatrix{a\\b}\\ &= 121 a^2 + 81 b^2 + 2c^2 ab\\ &=(11a)^2+(9b)^2+\frac{2c}{(11)(9)}(11a)(9b)\\ &= \alpha^2 + \beta^2 + \frac{2c}{(11)(9)} \alpha\beta. \end{aligned}

The last few steps in which $$\alpha=11a$$ and $$\beta=9b$$ were introduced weren't necessary, but they help to simplify the algebra. In particular, what we need to do next (in order to find bounds for $$c$$) is complete the square: this is the process emulating the derivation of the quadratic formula to which everyone is introduced in grade school. Writing

$$C = \frac{c}{(11)(9)},\tag{*}$$

we find

$$\alpha^2 + \beta^2 + \frac{2c^2}{(11)(9)} \alpha\beta = \alpha^2 + 2C\alpha\beta + \beta^2 = (\alpha+C\beta)^2+(1-C^2)\beta^2.$$

Because $$(\alpha+C\beta)^2$$ and $$\beta^2$$ are both squares, they are not negative. Therefore if $$1-C^2$$ also is non-negative, the entire right side is not negative and can be a valid variance. Conversely, if $$1-C^2$$ is negative, you could set $$\alpha=-c\beta$$ to obtain the value $$(1-C^2)\beta^2\lt 0$$ on the right hand side, which is invalid.

You therefore deduce (from these perfectly elementary algebraic considerations) that

If $$A$$ is a valid covariance matrix, then $$1-C^2$$ cannot be negative.

Equivalently, $$|C|\le 1,$$ which by $$(*)$$ means $$-(11)(9) \le c \le (11)(9).$$

There remains the question whether any such $$c$$ does correspond to an actual variance matrix. One way to show this is true is to find a random variable $$(X,Y)$$ with $$\mathbb A$$ as its covariance matrix. Here is one way (out of many).

I take it as given that you can construct independent random variables $$A$$ and $$B$$ having unit variances: that is, $$\operatorname{Var}(A)=\operatorname{Var}(B) = 1.$$ (For example, let $$(A,B)$$ take on the four values $$(\pm 1, \pm 1)$$ with equal probabilities of $$1/4$$ each.)

The independence implies $$\operatorname{Cov}(A,B)=0.$$ Given a number $$c$$ in the range $$-(11)(9)$$ to $$(11)(9),$$ define random variables

$$X = \sqrt{11^2-c^2/9^2}A + (c/9)B,\quad Y = 9B$$

(which is possible because $$11^2 - c^2/9^2\ge 0$$) and compute that the covariance matrix of $$(X,Y)$$ is precisely $$\mathbb A.$$

Finally, if you carry out the same analysis for any symmetric matrix $$\mathbb A = \pmatrix{a & b \\ b & d},$$ you will conclude three things:

1. $$a \ge 0.$$

2. $$d \ge 0.$$

3. $$ad - b^2 \ge 0.$$

These conditions characterize symmetric, positive semi-definite matrices. Any $$2\times 2$$ matrix satisfying these conditions indeed is a variance matrix. (Emulate the preceding construction.)

• It might be worth mentioning that $C$ here is the correlation and, as shown, is always between $-1$ and $+1$ Apr 17 at 12:00

An intuitive method to determine this answer quickly is to just remember that covariance matrices may be interpreted in the form

$$$$A = \begin{pmatrix} \sigma_1^2 & \rho_{12}\sigma_1\sigma_2 &\rho_{13}\sigma_1\sigma_3 & \cdots & \rho_{1n}\sigma_1 \sigma_n \\ & \sigma_2^2 & \rho_{23}\sigma_2\sigma_3 & \cdots & \rho_{2n}\sigma_2 \sigma_n \\ & & \sigma_3^2 & \cdots & \rho_{3n}\sigma_3 \sigma_n \\ & & & \ddots & \vdots \\ & & & & \sigma_n^2 \end{pmatrix}$$$$

where $$\rho_{ab} \in [-1,1]$$ is a Pearson Correlation Coefficient. In your case you have

\begin{align} \sigma_1^2 = 121 ,~~~ \sigma_2^2 = 81 ~\Longrightarrow ~ |c| \leq \sqrt{121\cdot 81} = 99 \end{align}

i.e. $$c \in [-99, 99]$$.

• +1 This is a great answer for readers familiar with this representation of covariance matrices. I don't think it would be remiss, though, to point out that when $n\gt 2$ the correlation coefficients are subject to additional restrictions: it doesn't suffice for them all just to lie between $-1$ and $1.$ The case $n=3$ is discussed at stats.stackexchange.com/questions/72790.
– whuber
Apr 17 at 18:05

$$A$$ is posdef so by Sylvesters criterion $$det(A) = 121 \cdot 81 - c^2 \geq 0$$. Thus, any $$c \in [-99, 99]$$ will produce a valid covariance matrix.

• Covariance matrices can be positive semi-definite right? Does the semi here change anything? Apr 16 at 15:23
• semi is the case when c is exactly 99 and det = 0. Apr 16 at 16:16
• @Hunaphu $c=+99$ and $c=-99$ both lead to zero determinant. If $-99 < c < +99$ then you would have a strictly positive definite matrix Apr 17 at 12:03
There are three main possibilities of note. One is that the variable are uncorrelated, in which case the off-diagonal entries are easy to calculate as 0. Another possibility is that you don't really have two different variables. $$y$$ is simply a scalar multiple of $$x$$ (i.e. perfect correlation). If $$y= c x$$, then $$\sigma_{xy} =\sigma_{x}\sigma_{xy}=99$$. We get a third possibility in noting that the above assumes $$c>0$$. For $$c<0$$, we get $$\sigma_{xy} =-99$$.
Geometrically, the covariance between two vectors is the product of their lengths times the cosine of the angle between them. Since the cosine varies from $$-1$$ to $$1$$, the covariance ranges from the product of their lengths to the negative of the product.
Another approach is to consider $$z_1 = \frac{x-\mu_{x}}{\sigma_{x}}$$ and $$z_2 = \frac{y-\mu_y}{\sigma_{y}}$$. $$\sigma_{xy} = \sigma_{(\sigma_x z_1)(\sigma_y z_2)}=\sigma_x \sigma_y \sigma_{z_1z_2}=99\sigma_{z_1z_2}$$ and $$\sigma_{z_1z_2}$$ is simply the correlation between $$x$$ and $$y$$, which ranges from $$-1$$ to $$1$$.