2
$\begingroup$

Assuming there is enough data, and all predictors and independent variable are positively correlated, in other words, every possible pairwise correlation is positive. Is it possible to end up with some negative coefficients in a multi-linear model fit? All variables are defined on a continuous scale.

$$ y = \alpha x_1 + \beta x_2 $$ where: $cor(y, x1) > 0$, $cor(y, x2) > 0$, $cor(x1,x2) > 0$

Given the above can either a or b end up being negative?

$\endgroup$
6
  • 2
    $\begingroup$ Yes, it's possible. Look for confounding. $\endgroup$
    – Aksakal
    Oct 24, 2016 at 17:36
  • 1
    $\begingroup$ Would you mind being more precise? As I see it confounding relates to causal modelling and change of coefficients on including/excluding a possible variable that may be a common cause. However here the data is given, there is no attempt of inferring causal relationships, the predictors are fixed. $\endgroup$
    – Anton
    Oct 24, 2016 at 17:43
  • 3
    $\begingroup$ It's also called a suppressor effect. $\endgroup$ Oct 24, 2016 at 17:45
  • 3
    $\begingroup$ I was referring to the mechanism that drives confounding. Look at this example: ryerson.ca/~tsly/confounding.htm $\endgroup$
    – Aksakal
    Oct 24, 2016 at 17:46
  • 2
    $\begingroup$ Pairwise correlations aren't accurate reflections of predictor behavior in a multiple regression. Partial or semi-partial correlations are the better metrics. $\endgroup$ Oct 24, 2016 at 18:22

2 Answers 2

2
$\begingroup$

Here's example:

y    x1 x2
-1.1    0.1 3
-0.1    0.2 1
-0.3    0.3 3
-0.1    0.4 5
0.5  0.5    4
1.2  0.6    6
2.3  0.7    8
2.9  0.8    7

Here's how I constructed it: $$y = x_1 - x_2/2 + x_1*x_2$$

You can see how all three pairwise correlations are positive, but if you regress on two variables $y\sim x_1+x_2$, you'll get the negative slope on the second variable, as you should.

$\endgroup$
2
$\begingroup$

Here's a smaller example:

  • $x_1 = (0, 2, 1)$
  • $x_2 = (0, 1, 1)$
  • $y = (0, 1, 2)$

All three correlations are at least $\tfrac{1}{2}$, but the coefficient of $x_1$ in the regression is $-1$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.