I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation coefficient for the first row and column is lookalike [1,0.2,-0.8,0.01,0.9], should I expect a high linear coefficient for variables $x_2$ and $x_4$?
4
-
2$\begingroup$ No, not necessarily, correlation is univariate, (multiple) linear regression is not, and the results might differ. $\endgroup$– user2974951May 18, 2022 at 6:19
-
1$\begingroup$ Not only shouldn't you have any expectations for the regression coefficients, you also shouldn't draw any conclusions from a model set up with these variables, since you are violating one of the main prerequisites for linear regression. See towardsdatascience.com/…, r-statistics.co/Assumptions-of-Linear-Regression.html $\endgroup$– rumtschoMay 18, 2022 at 7:10
-
1$\begingroup$ You should expect a negative correlation between $x_2$ and $x_4$, I think between $-0.458$ and $-0.982$. But there will be examples where the linear regression coefficients appear small in magnitude, for example when the variances of $x_2$ and $x_4$ are much larger than the variance of $y$; even if that is not the case, there will be other examples where at least one has a small linear regression coefficient $\endgroup$– HenryMay 18, 2022 at 8:45
-
1$\begingroup$ @rumtscho Lack of feature multicollinearity is not an assumption of linear regression, and that article you've linked glosses over complex issues of modeling. See, for instance, my answer here about why discarding correlated features can be dangerous. // It is a common misconception to think that lacking feature correlation is an assumption of linear regression; I think this comes from misinterpreting the lack of error correlation in the Gauss-Markov theorem. $\endgroup$– DaveMay 24, 2022 at 13:06
Add a comment
|