I created a model, and in some cases I have regression coefficients between 2 variables that are slightly higher than correlation coefficients. Is that normal?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ A related and relevant discussion stats.stackexchange.com/questions/32464/… $\endgroup$– KortchnoiCommented Apr 27, 2020 at 8:47
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
3
In simple words, regression coefficient is the amount of change in the dependent variable when the independent variable(predictor) changes by one unit. In the case of linear regression, the R is the correlation coefficient is the measure of correlation between actual values of the dependent variable and the predicted values of the dependent variable using the regression model. Thus, there could not be any comparison between the regression coefficient and correlation coefficient.
If the question is about the coefficient of correlation between the two variables (independent and dependent variables), it is the degree to which both the variables are varying together (when one is varying in one direction, the extent to which the other one is varying in same direction or in the opposite direction). This value is completely different from R of the regression model.
The correlation coefficient ranges from -1 to 1, where the value closer to -1 denotes high negative correlation and closer to 1 denotes high positive correlation.On the other side, there is no fixed range for regression coefficient. It depends on the amount to which the predictor influences the dependent variable. It depends on how the predictor and the dependent variables are scaled.
-
$\begingroup$ amazing, it's much clearer now. Thank you very much! $\endgroup$ Commented Apr 26, 2020 at 13:35
-
-
1$\begingroup$ You define the correlation coefficient for another pair of variables ($y$ and $\hat y$) than corresponds to the regression coefficient ($y$ and $x$). I doubt this makes sense in the context of the question. $\endgroup$ Commented Apr 27, 2020 at 14:05
$\begingroup$
$\endgroup$
1
For simple linear regression there is a relationship between slope and correlation:
$\hat\beta_1 = r_{x,y}{s_y\over s_x}$
So the relationship of $\hat\beta_1$ and $r_{xy}$ is entirely dependent on the standard deviations of x and y, and, by rescaling variables, can be pretty much any value.
-
1$\begingroup$ As an example, suppose you were interested in the relation between home price and the size of the house. Doing the regression using square feet will give you a much bigger B1 than if you did the regression using square inches, but the correlation between price and size of the house would be the same regardless of the unit used. $\endgroup$– epsCommented Apr 27, 2020 at 20:11
$\begingroup$
$\endgroup$
2
Yes, it's absolutely normal.
The size of your regression coefficients depends on the units of measurement of your explanatory variables. i.e. regression coefficients will be larger if height is measured in meters $(x_i = 1.8$ m$)$ than if it's measured in centimetres $(x_i=180$ cm$)$.
Correlation, on the other hand, is a standardized metric. It does not depend on the unit of measurement.
Hence you can make your regression coefficients arbitrarily small or large by changing unit of measurement - but this has no effect on correlation.
-
1$\begingroup$ While true, this seems more like an explanation of the difference between covariance and correlation. $\endgroup$ Commented Apr 27, 2020 at 1:12
-
1$\begingroup$ A different way to explain is that regression coefficients and correlation coefficients aren't even comparable unless a predictor is standardized, any more than it makes sense to compare a length and an area numerically, because they have different units. $\endgroup$– Nick CoxCommented Apr 27, 2020 at 8:09
$\begingroup$
$\endgroup$
1
The coefficient of determination ($R^2$) in a linear regression model is the square of the Pearson correlation coefficient ($r$). The regression coefficients themselves are more weakly related to $r$, being driven more by the variation in X and Y. So yes, your result is entirely normal.
-
$\begingroup$ You define the correlation coefficient for another pair of variables ($y$ and $\hat y$) than corresponds to the regression coefficient ($y$ and $x$). I doubt this makes sense in the context of the question. $\endgroup$ Commented Apr 27, 2020 at 14:07