Linear regression: positively correlated predictors and negative coefficients

Question

Assuming there is enough data, and all predictors and independent variable are positively correlated, in other words, every possible pairwise correlation is positive. Is it possible to end up with some negative coefficients in a multi-linear model fit? All variables are defined on a continuous scale.

$$ y = \alpha x_1 + \beta x_2 $$ where: $cor(y, x1) > 0$, $cor(y, x2) > 0$, $cor(x1,x2) > 0$

Given the above can either a or b end up being negative?

Answer 1

Here's example:

y    x1 x2
-1.1    0.1 3
-0.1    0.2 1
-0.3    0.3 3
-0.1    0.4 5
0.5  0.5    4
1.2  0.6    6
2.3  0.7    8
2.9  0.8    7

Here's how I constructed it: $$y = x_1 - x_2/2 + x_1*x_2$$

You can see how all three pairwise correlations are positive, but if you regress on two variables $y\sim x_1+x_2$, you'll get the negative slope on the second variable, as you should.

Answer 2

Here's a smaller example:

• $x_1 = (0, 2, 1)$
• $x_2 = (0, 1, 1)$
• $y = (0, 1, 2)$

All three correlations are at least $\tfrac{1}{2}$, but the coefficient of $x_1$ in the regression is $-1$.