2
$\begingroup$

I ran a lasso. I then used the variables that had non-zero coefficients in a linear regression. I would've expected the variables with the largest coefficients in the lasso to have the smallest P-values in the regression output, but this was not the case. In fact, the variable with the largest coefficient in lasso was not significant at all in my regression.

Can someone provide the intuition and/or mathematical reasoning for why this happens?

My leading guess is that it could be because of correlations between the variables used in the regression, leading to multicollinearity? If so, why doesn't multicollinearity matter for lasso?

The top answer under this question begins to answer my question but doesn't provide much of the mathematical intuition: Lasso regression coefficients values

$\endgroup$
8
  • $\begingroup$ Did you standardize your variables? $\endgroup$ – whuber Mar 1 '20 at 2:21
  • $\begingroup$ Yes, they are all on a scale of 1-7. I removed all variables that were not 1-7. $\endgroup$ – melbez Mar 1 '20 at 3:26
  • 1
    $\begingroup$ @melbez That isn't standardizing. Can you please transform your predictors so that each column has mean 0 and unit standard deviation. $\endgroup$ – Demetri Pananos Mar 1 '20 at 3:52
  • $\begingroup$ Ah I see! What would that accomplish in relation to the question I asked? $\endgroup$ – melbez Mar 1 '20 at 3:55
  • 1
    $\begingroup$ @melbez For reasons I won't get into here, it is better to perform the lasso when the data are standardized in this fashion. $\endgroup$ – Demetri Pananos Mar 1 '20 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.