I'm curious to know, if three random variables have the same variance, what will the co-variances look like? Can somebody help me to figure it out?


The covariances will be the common variance mutliplied by the correlations between the variables, which will be between $-1$ and $1$, though there are some restrictions on the possible values the set of three correlations might take (they can't all be $-1$ at the same time, for example).

So if the variables are $Y_1,Y_2$ and $Y_3$, the common variance is $\sigma^2$ and the three correlations are $\rho_{12},\rho_{13}$ and $\rho_{23}$, then the covariance $c_{ij} = \sigma^2 \rho_{ij}$ for each $i,j$ combination.

The set of possible values for the three correlations is such that either the correlation matrix or the covariance matrix (either implies the other given the variances are all positive) is positive semi-definite. A symmetric $n × n$ real matrix $A$ is positive semi-definite if $x^TAx \geq 0$ for every non-zero column vector $x$ of $n$ real numbers.

(In your case, $n=3$ of course.)

So, for an example of a correlation matrix that cannot occur, consider the case where all the correlations $\rho_{12},\rho_{13}$ and $\rho_{23}$ are $-0.8$. Then if you take the vector $x$ above to be all $1$'s, $x^TAx = -1.8$, so there's at least one $x$ for which $x^TAx$ isn't at least $0$, and so the matrix $A$ isn't positive semi-definite, and cannot be a correlation matrix.

In R:

> A
     [,1] [,2] [,3]
[1,]  1.0 -0.8 -0.8
[2,] -0.8  1.0 -0.8
[3,] -0.8 -0.8  1.0

> x
[1,]    1
[2,]    1
[3,]    1

> t(x) %*% A %*% x   #  x' A x
[1,] -1.8

So if we try to do anything that relies on it being positive semi-definite, it will fail:

> chol(A)
Error in chol.default(A) : 
  the leading minor of order 3 is not positive definite

[Why does that definition of what's a possible correlation matrix work? Consider that we define a new random variable, $Z = x^TY$. Then $\text{Var}(Z) = x^T\text{Var}(Y)x = \sigma^2 x^TAx$. Clearly, if $x^TAx$ could be negative, there'd be a random variable, $Z$ with a negative variance. So we need $x^TAx\geq 0$ for every possible $x$.]

edit: some more-or-less related questions:

The bound on a common correlation of three variables (the bounds on $\rho$ for the case $\rho_{12}=\rho_{13}=\rho_{23}=\rho$) is discussed in this question.

If you specify two correlations and look at the possible values of the third, there's some relevant discussion in this question.

  • $\begingroup$ In my answer to the question on bounds for a common correlation value, I had pointed out that the average correlation of $n$ random variables is lower bounded by $-\frac{1}{n-1}$ which equals $-\frac 12$ when $n = 3$. This gives an easier answer to your example of "common correlation value cannot be $-0.8$". $\endgroup$ – Dilip Sarwate Nov 10 '15 at 5:05
  • $\begingroup$ @Dilip My answer already contains a link to that question $\endgroup$ – Glen_b -Reinstate Monica Nov 10 '15 at 5:07
  • $\begingroup$ Yes, I know that your answer contains a link to the question. I am just saying that lots of information in the various answers to the two questions you link to is very relevant to the question asked here. That is, those two questions are better described as "closely related" rather than "more-or-less related" $\endgroup$ – Dilip Sarwate Nov 10 '15 at 5:11
  • $\begingroup$ @Dilip I think my statement "The bound on a common correlation of three variables (the bounds on ρ for the case ρ12=ρ13=ρ23=ρ) is discussed in this question." conveys a much stronger sense of being related than you suggest by "more-or-less related"; I see it as saying it directly responds to that specific issue. What would you like it to say? $\endgroup$ – Glen_b -Reinstate Monica Nov 10 '15 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.