I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation coefficient for the first row and column is lookalike [1,0.2,-0.8,0.01,0.9], should I expect a high linear coefficient for variables $x_2$ and $x_4$?

  • 2
    $\begingroup$ No, not necessarily, correlation is univariate, (multiple) linear regression is not, and the results might differ. $\endgroup$ May 18, 2022 at 6:19
  • 1
    $\begingroup$ Not only shouldn't you have any expectations for the regression coefficients, you also shouldn't draw any conclusions from a model set up with these variables, since you are violating one of the main prerequisites for linear regression. See towardsdatascience.com/…, r-statistics.co/Assumptions-of-Linear-Regression.html $\endgroup$
    – rumtscho
    May 18, 2022 at 7:10
  • 1
    $\begingroup$ You should expect a negative correlation between $x_2$ and $x_4$, I think between $-0.458$ and $-0.982$. But there will be examples where the linear regression coefficients appear small in magnitude, for example when the variances of $x_2$ and $x_4$ are much larger than the variance of $y$; even if that is not the case, there will be other examples where at least one has a small linear regression coefficient $\endgroup$
    – Henry
    May 18, 2022 at 8:45
  • 1
    $\begingroup$ @rumtscho Lack of feature multicollinearity is not an assumption of linear regression, and that article you've linked glosses over complex issues of modeling. See, for instance, my answer here about why discarding correlated features can be dangerous. // It is a common misconception to think that lacking feature correlation is an assumption of linear regression; I think this comes from misinterpreting the lack of error correlation in the Gauss-Markov theorem. $\endgroup$
    – Dave
    May 24, 2022 at 13:06


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.