Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

The sign of a regression coefficient will always be the same as the sign of the correlation coefficient between the corresponding predictor variable and the dependent variable.

  • Is the above statement always correct?
  • If no, what can be the reason if the signs differ?
share|improve this question
add comment

3 Answers 3

The statement is true iff you included only one explanatory variable. I suppose you know that the regression coefficent in such a regression is just $r_{x,y}\frac{s_y}{s_x}$, where $r_{x,y}$ is the correlation coefficient, and $s_y$ and $s_x$ are the standard deviations of $y$ and $x$ respectively. Since standard deviations cannot be negative, the ratio $\frac{s_y}{s_x}$ will always be positive, so the sign of the regression coefficient is only determined by the sign of the correlation.

However, things are very different when you add more than one explanatory variable.

share|improve this answer
add comment

Expanding on @Maarten 's answer:

Suppose you are predicting the damage done by a fire. First, you look at one IV: Number of firefighters called to the scene. To your surprise, you find a strong positive relationship: More firefighters, more damage. Then you think of adding "size of fire" and add that to the equation; the relationship between firefighters and damage will now be negative. That's mediation.

Another way this can happen is moderation: Where the two IV interact.

share|improve this answer
add comment

Assume you have a regression with two regressors (plus a constant), $$y_i = a + b_1x_{1i}+b_2x_{2i} + u_i$$ Then if you work the normal equations (a bit tedious) you will find that

$$\hat b_1 = \frac {\operatorname {\hat Var}(X_2)\cdot \operatorname {\hat Cov}(Y,X_1) - \operatorname {\hat Cov}(X_1,X_2)\cdot \operatorname {\hat Cov}(Y,X_2)}{\operatorname {\hat Var}(X_1)\cdot\operatorname {\hat Var}(X_2)\cdot [1-\hat \rho_{1,2}^2]} $$

where the hat indicates sample variances (without the bias correction term), and covariances, and $\hat \rho_{1,2}$ is the sample correlation coefficient between the two regressors.

The denominator is always positive, so the sign of the estimated coefficient depends on the numerator. Then if you have (for example)

$$0 < \operatorname {\hat Cov}(Y,X_1) < \frac {\operatorname {\hat Cov}(X_1,X_2)\cdot \operatorname {\hat Cov}(Y,X_2)}{\operatorname {\hat Var}(X_2)}$$

which is perfectly possible, then you will have positive pair-wise correlation between the dependent variable and regressor $X_1$, but negative coefficient of this regressor in the context of multiple regression. In other words, if one examines the dependent variable and regressor $X_1$ alone, they tend to move together (i.e. one will obtain a positive coefficient in the context of simple regression), but if regressor $X_2$ is present the marginal effect of $X_1$ on the dependent variable emerges as negative. This is an instance of the famous "sign reversal paradox", which is not really a paradox. Intuition (for this case)? If $X_2$ strongly correlates positively with the dependent variable and $X_1$, then in the simple regression the apparent positive relation between $Y$ and $X_1$ is due to the underlying effect of $X_2$ which is absent. When $X_2$ enters the specification, it takes on this positive effect, and "reveals" that the "pure" effect of $X_1$ on the dependent variable is, after all, negative. Note that the simple regression here would constitute a case of "omitted variables bias" in the estimation, since $X_2$ does belong to the specification.

In some fields, $X_2$ is called a "confounder". Analogous results hold of course for the other coefficient, or for more than two regressors.

See also Positive correlation and negative regressor coefficient sign

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.