22
$\begingroup$

A is positively related to B.

C is the outcome of A and B, but the effect of A on C is negative and the effect of B on C is positive.

Can this happen?

$\endgroup$
  • $\begingroup$ This is a relation in the model in SEM $\endgroup$ – Reen Aug 9 '16 at 19:10
  • 1
    $\begingroup$ stats.stackexchange.com/q/33888/3277 is a closely related question. Not identical, but the answers could be extrapolated over to here. $\endgroup$ – ttnphns Aug 25 '16 at 20:23
43
$\begingroup$

The other answers are truly marvelous - they give real life examples.

I want to explain why this can happen despite our intuition to the contrary.

See this geometrically!

Correlation is the cosine of the angle between the vectors. Essentially, you are asking whether it is possible that

  • $A$ makes an acute angle with $B$ (positive correlation)
  • $B$ makes an acute angle with $C$ (positive correlation)
  • $A$ makes an obtuse angle with $C$ (negative correlation)

Yes, of course:

enter image description here

In this example ($\rho$ denotes correlation):

  • $A=(0.6,0.8)$
  • $B=(1,0)$
  • $C=(0.6,-0.8)$
  • $\rho(A,B)=0.6>0$
  • $\rho(B,C)=0.6>0$
  • $\rho(A,C)=-0.28<0$

Your Intuition is Right!

However, your surprise is not misplaced.

The angle between vectors is a distance metric on the unit sphere, so it satisfies the triangle inequality:

$$\measuredangle AB \le \measuredangle AC + \measuredangle BC$$

thus, since $\cos \measuredangle AB = \rho(A,B)$,

$$\arccos\rho(A,B) \le \arccos\rho(A,C) + \arccos\rho(B,C) $$

therefore (since $\cos$ is decreasing on $[0,\pi]$)

$$\rho(A,B)\ge\rho(A,C)\times\rho(B,C) - \sqrt{(1-\rho^2(A,C))\times(1-\rho^2(B,C))} $$

So,

  • if $\rho(A,C)=\rho(B,C)=0.9$, then $\rho(A,B)\ge 0.62$
  • if $\rho(A,C)=\rho(B,C)=0.95$, then $\rho(A,B)\ge 0.805$
  • if $\rho(A,C)=\rho(B,C)=0.99$, then $\rho(A,B)\ge 0.9602$
$\endgroup$
32
$\begingroup$

Yes, two co-occuring conditions can have opposite effects.

For example:

  • Making outrageous statements (A) is positively related to being entertaining (B).
  • Making outrageous statements (A) has a negative effect on winning elections (C).
  • Being entertaining (B) has a positive effect on winning elections (C).
$\endgroup$
  • 20
    $\begingroup$ We have the best answers. The best. Everybody says so. $\endgroup$ – Matthew Drury Aug 9 '16 at 19:28
  • 1
    $\begingroup$ Although I agree with this political opinion, I think it's bad form to use an answer on this site as a vehicle for an irrelevant political opinion. $\endgroup$ – Kodiologist Aug 9 '16 at 19:58
  • 14
    $\begingroup$ @Kodiologist This answer doesn't take a stance on any candidate or any issue. It makes the fairly unremarkable (imho) observations that: (1) entertaining candidates have an advantage (eg. Ronald Reagan, Bill Clinton, Willie Brown) and (2) highly provocative statements tend to hurt more than they help (which is why politicians tend not to make these types of statements). If this is a no fun zone, I can take it down, but I think what I wrote is incredibly benign and uncontroversial. $\endgroup$ – Matthew Gunn Aug 9 '16 at 21:40
  • 19
    $\begingroup$ I don't see any direct political references in the answer. There may be an implied reference, but I don't think that in any way impacts the validity or suitability of the answer. $\endgroup$ – Glen_b Aug 9 '16 at 22:26
28
$\begingroup$

I've heard this car analogy which applies well to the question:

  • Driving uphill (A) is positively related to the driver stepping on the gas (B)
  • Driving uphill (A) has a negative effect on vehicle speed (C)
  • Stepping on the gas (B) has a positive effect on vehicle speed (C)

The key here is the driver's intention to maintain a constant speed (C), therefore the positive correlation between A and B naturally follows from that intention. You can construct endless examples of A, B, C with this relationship thus.

The analogy comes from an interpretation of Milton Friedman's Thermostat and comes from an interesting analysis of monetary policy and econometrics, but that's irrelevant to the question.

$\endgroup$
  • 2
    $\begingroup$ Nice example. However, I'm not sure that you are using the terms 'positively related' and 'negatively related' as statistical relationships (e.g. correlation), which I guess is what the op means. $\endgroup$ – Lior Kogan Aug 10 '16 at 8:02
8
$\begingroup$

Yes, this is trivial to demonstrate with a simulation:

Simulate 2 variables, A and B that are positively correlated:

> require(MASS)
> set.seed(1)
> Sigma <- matrix(c(10,3,3,2),2,2)
> dt <- data.frame(mvrnorm(n = 1000, rep(0, 2), Sigma))
> names(dt) <- c("A","B")
> cor(dt)

          A         B
A 1.0000000 0.6707593
B 0.6707593 1.0000000

Create variable C:

> dt$C <- dt$A - dt$B + rnorm(1000,0,5)

Behold:

> (lm(C~A+B,data=dt))

Coefficients:
(Intercept)            A            B  
    0.03248      0.98587     -1.05113  

Edit: Alternatively (as suggested by Kodiologist), just simulating from a multivariate normal such that $\operatorname{cor}(A,B) > 0$, $\operatorname{cor}(A,C) > 0$ and $\operatorname{cor}(B,C) < 0$

> set.seed(1)
> Sigma <- matrix(c(1,0.5,0.5,0.5,1,-0.5,0.5,-0.5,1),3,3)
> dt <- data.frame(mvrnorm(n = 1000, rep(0,3), Sigma, empirical=TRUE))
> names(dt) <- c("A","B","C")
> cor(dt)
    A    B    C
A 1.0  0.5  0.5
B 0.5  1.0 -0.5
C 0.5 -0.5  1.0
$\endgroup$
  • $\begingroup$ I think it's better to look at cor(C, A) and cor(C, B) than lm(C ~ A + B) here. We're interested in, e.g., the uncontrolled relationship of A and C rather than this relationship controlled for B. $\endgroup$ – Kodiologist Aug 9 '16 at 20:01
  • $\begingroup$ @Kodiologist the OP says in their comment that the context is a SEM, which would imply a linear regression, I think. $\endgroup$ – Robert Long Aug 9 '16 at 20:06
  • $\begingroup$ @Kodiologist see the update to my answer :) $\endgroup$ – Robert Long Aug 9 '16 at 20:23
0
$\begingroup$

$$ C = mB + n (A-proj_B(A)) $$

then $$ \left<C,A\right> = m\left<B,A\right> + n\left<A,A\right> -n \left<B,A\right> $$

Then covariance between C and A could be negative in two conditions:

  1. $n>m,\ \left<A,A\right> < \left<B,A\right> (n-m)/n $
  2. $n<-m,\ \left<A,A\right> > \left<B,A\right> (n-m)/n$
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.