For simple linear regression, the regression coefficient's sign and the correlation coefficient's (between independent and dependent variable) sign should be matching or not?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
Yes, in the case of a simple linear regression, they should be matching! If $x$ increases and $y$ increases, that means they are positively correlated. In that case, the slope (coefficient of $x$) will be positive. If $x$ increases and $y$ decreases, we always have a negative correlation and a negative slope. Also $cor(y,x) = \sqrt{R^2}$.
answered Jul 27, 2017 at 22:42
-
$\begingroup$ If it's not matching, then what could be the reason? Can collinearity couse the issue? $\endgroup$ Commented Jul 27, 2017 at 22:56
-
$\begingroup$ When you said simple linear regression, I assumed you meant with only one independent variable. In that case, it should always match. If it's multiple independent variables, then no, it really doesn't have to match. A correlation is a measure of the linear relationship between 2 variables, while a multiple regression is a measure of the linear relationship between $y$ and multiple $x$s. $\endgroup$ Commented Jul 27, 2017 at 23:02
-
$\begingroup$ But the coefficient for a variable should be independent of the other variables, since its the change in outcome with a unit increase in the value of that particular variable. Please correct me if I am wrong. $\endgroup$ Commented Jul 27, 2017 at 23:06
-
$\begingroup$ It is not independent. Say you regress $y$ on $x_1$ and $x_2$, and you get significant coefficients $\hat{\beta_1}$ and $\hat{\beta_2}$. Now repeat by omitting $x_2$. Now you have $\hat{\beta_1'}$, which is equal to $\hat{\beta_1}$ plus some bias. The bias is proportional to the correlation between $x_1$ and $x_2$, so the coefficient is not independent of the other variables (to be more precise, the variables are not independent of each other). In any case, none of these are applicable in a simple linear regression setting, was asked in the original question. $\endgroup$ Commented Jul 27, 2017 at 23:19
-
1$\begingroup$ This answer is good, but the final formula is incorrect because $\sqrt{R^2}=|R| \ge 0$ whereas the correlation can be negative. $\endgroup$– whuber ♦Commented Dec 15, 2019 at 14:50