If A and B are correlated with C, why are A and B not necessarily correlated?

Question

I know empirically that is the case. I have just developed models that run into this conundrum. I also suspect it is not necessarily a yes/no answer. I mean by that if both A and B are correlated with C, this may have some implication regarding the correlation between A and B. But, this implication may be weak. It may be just a sign direction and nothing else.

Here is what I mean... Let's say A and B both have a 0.5 correlation with C. Given that, the correlation between A and B could well be 1.0. I think it also could be 0.5 or even lower. But, I think it is unlikely that it would be negative. Do you agree with that?

Also, is there an implication if you are considering the standard Pearson Correlation Coefficient or instead the Spearman (rank) Correlation Coefficient? My recent empirical observations were associated with the Spearman Correlation Coefficient.

Answer 1

Because correlation is a mathematical property of multivariate distributions, some insight can be had purely through calculations, regardless of the statistical genesis of those distributions.

For the Pearson correlations, consider multinormal variables $X$, $Y$, $Z$. These are useful to work with because any non-negative definite matrix actually is the covariance matrix of some multinormal distributions, thereby resolving the existence question. If we stick to matrices with $1$ on the diagonal, the off-diagonal entries of the covariance matrix will be their correlations. Writing the correlation of $X$ and $Y$ as $\rho$, the correlation of $Y$ and $Z$ as $\tau$, and the correlation of $X$ and $Z$ as $\sigma$, we compute that

• $1 + 2 \rho \sigma \tau - \left(\rho^2 + \sigma^2 + \tau^2\right) \ge 0$ (because this is the determinant of the correlation matrix and it cannot be negative).

• When $\sigma = 0$ this implies that $\rho^2 + \tau^2 \le 1$. To put it another way: when both $\rho$ and $\tau$ are large in magnitude, $X$ and $Z$ must have nonzero correlation.

• If $\rho^2 = \tau^2 = 1/2$, then any non-negative value of $\sigma$ (between $0$ and $1$ of course) is possible.

• When $\rho^2 + \tau^2 \lt 1$, negative values of $\sigma$ are allowable. For example, when $\rho = \tau = 1/2$, $\sigma$ can be anywhere between $-1/2$ and $1$.

These considerations imply there are indeed some constraints on the mutual correlations. The constraints (which depend only on the non-negative definiteness of the correlation matrix, not on the actual distributions of the variables) can be tightened depending on assumptions about the univariate distributions. For instance, it's easy to see (and to prove) that when the distributions of $X$ and $Y$ are not in the same location-scale family, their correlations must be strictly less than $1$ in size. (Proof: a correlation of $\pm 1$ implies $X$ and $Y$ are linearly related a.s.)

As far as Spearman rank correlations go, consider three trivariate observations $(1,1,2)$, $(2,3,1)$, and $(3,2,3)$ of $(X, Y, Z)$. Their mutual rank correlations are $1/2$, $1/2$, and $-1/2$. Thus even the sign of the rank correlation of $Y$ and $Z$ can be the reverse of the signs of the correlations of $X$ and $Y$ and $X$ and $Z$.

Answer 2

I'm on an annual fishing trip right now. There is a correlation between the time of day I fish and the amount of fish I catch. There is also a correlation between the size of the bait I use and the amount of fish I catch. There is no correlation between the size of the bait and the time of day.

Answer 3

As an add-on to whuber's answer: The presented formula

$1 + 2 \rho \sigma \tau - \left(\rho^2 + \sigma^2 + \tau^2\right) \ge 0$.

can be transformed into following inequality (Olkin, 1981):

$\sigma\tau - \sqrt{(1-\sigma^2)(1-\tau^2)} \le \rho \le \sigma\tau + \sqrt{(1-\sigma^2)(1-\tau^2)}$

A graphical representation of the upper and lower limits for $\rho$ looks like:

---

Olkin, I. (1981). Range restrictions for product-moment correlation matrices. Psychometrika, 46, 469-472. doi:10.1007/BF02293804

Answer 4

Correlation is the cosine of the angle between two vectors. In the situation described, (A,B,C) is a triple of observations, made n times, each observation being a real number. The correlation between A and B is the cosine of the angle between $V_A=A-E(A)$ and $V_B=B-E(B)$ as measured in n-dimensional euclidean space. So our situation reduces to considering 3 vectors $V_A$, $V_B$ and $V_C$ in n dimensional space. We have 3 pairs of vectors and therefore 3 angles. If two of the angles are small (high correlation) then the third one will also be small. But to say "correlated" is not much of a restriction: it means that the angle is between 0 and $\pi/2$. In general this gives no restriction at all on the third angle. Putting it another way, start with any angle less than $\pi$ between $V_A$ and $V_B$ (any correlation except -1). Let $V_C$ bisect the angle between $V_A$ and $V_B$. Then C will be correlated with both A and B.

Answer 5

I will leave the statistical demonstration to those who are better suited than me for it... but intuitively say that event A generates a process X that contributes to the generation of event C. Then A is correlated to C (through X). B, on the other hand generates Y, that also shapes C. Therefore A is correlated to C, B is correlated to C but A and B are not correlated.

Answer 6

I think it's better to ask "why SHOULD they be correlated?" or, perhaps "Why should have any particular correlation?"

The following R code shows a case where x1 and x2 are both correlated with Y, but have 0 correlation with each other

    x1 <- rnorm(100)
    x2  <- rnorm(100)
    y <- 3*x1 + 2*x2 + rnorm(100, 0, .3)
    
    cor(x1,y)
    cor(x2,y)
    cor(x1,x2)

The correlation with $Y$ can be made stronger by reducing the .3 to .1 or whatever

Answer 7

For those who want some intuition, a correlation can be seen as a cosine of some angle. So, consider three vectors in 3D, let say A, B, and C, each corresponding to one variable. The question is to determine the range of possible angles between A and C when the angle between A and B as well as the angle between B et C are known. For that, you can play with an online tool without installing any software. Just go to the page http://www.montefiore.ulg.ac.be/~pierard/chained_correlations.php

Answer 8

Lets take one example:

A={x1,x2,x3,x4,x5,x6,x7,x8,x9}

B={x1,x2,x3,0,0,0,0,0,0}

C={0,0,0,x4,x5,x6,0,0,0}

For some x, A and B will have significant correlation, similarly A and C will also have significant correlation but correlation of B and C won't be significant.

So, It's not necessarily true that if A and B correlated and A and C are correlated then B and C are also correlated.

Note: For deep understanding, Please think this example on large data.

Answer 9

Because correlation is not the same as causation.

If A has a causal effect on B (resulting in a positive correlation between A and B), and B has a causal effect on C (resulting in a positive correlation between B and C), then A has a causal effect on C via B (and there will be a positive correlation between A and C).

You might illustrate this causal structure the following way:

A -> B -> C

If we simply specify that B is correlated with both A and C, this might be due to the underlying causal structure specified above, in which case A and C will be correlated. However, specifying the correlations generally under-constrains the causal structure. Another possible causal structure that could produce the desired correlations is one where A and C are totally independent but both have a causal influence on B (producing the desired positive correlation between both B and A, and B and C, without requiring any correlation between A and C).

You might illustrate this causal structure the following way:

A -> B <- C