2
$\begingroup$

Assuming there is enough data, and all predictors and independent variable are positively correlated, in other words, every possible pairwise correlation is positive. Is it possible to end up with some negative coefficients in a multi-linear model fit? All variables are defined on a continuous scale.

$$ y = \alpha x_1 + \beta x_2 $$ where: $cor(y, x1) > 0$, $cor(y, x2) > 0$, $cor(x1,x2) > 0$

Given the above can either a or b end up being negative?

$\endgroup$
  • 2
    $\begingroup$ Yes, it's possible. Look for confounding. $\endgroup$ – Aksakal Oct 24 '16 at 17:36
  • 1
    $\begingroup$ Would you mind being more precise? As I see it confounding relates to causal modelling and change of coefficients on including/excluding a possible variable that may be a common cause. However here the data is given, there is no attempt of inferring causal relationships, the predictors are fixed. $\endgroup$ – Anton Oct 24 '16 at 17:43
  • 3
    $\begingroup$ It's also called a suppressor effect. $\endgroup$ – Jeremy Miles Oct 24 '16 at 17:45
  • 3
    $\begingroup$ I was referring to the mechanism that drives confounding. Look at this example: ryerson.ca/~tsly/confounding.htm $\endgroup$ – Aksakal Oct 24 '16 at 17:46
  • 2
    $\begingroup$ Pairwise correlations aren't accurate reflections of predictor behavior in a multiple regression. Partial or semi-partial correlations are the better metrics. $\endgroup$ – Mike Hunter Oct 24 '16 at 18:22
2
$\begingroup$

Here's example:

y    x1 x2
-1.1    0.1 3
-0.1    0.2 1
-0.3    0.3 3
-0.1    0.4 5
0.5  0.5    4
1.2  0.6    6
2.3  0.7    8
2.9  0.8    7

Here's how I constructed it: $$y = x_1 - x_2/2 + x_1*x_2$$

You can see how all three pairwise correlations are positive, but if you regress on two variables $y\sim x_1+x_2$, you'll get the negative slope on the second variable, as you should.

$\endgroup$
2
$\begingroup$

Here's a smaller example:

  • $x_1 = (0, 2, 1)$
  • $x_2 = (0, 1, 1)$
  • $y = (0, 1, 2)$

All three correlations are at least $\tfrac{1}{2}$, but the coefficient of $x_1$ in the regression is $-1$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.