0
$\begingroup$

In the case of the ridge estimator, we can interpret the shrinkage induced by the ridge estimator to be at its most extreme when a predictor has low variance. High-variance predictors provide the most information/signal about our response, and low-variance ones contribute to "noise" which should be shrunk down - reminiscent of a continuous version of PCA here.

Is there a similar interpretation to the lasso? That is, how does lasso choose which covariates to shrink down and truncate and what does that imply about them? If a predictor is dropped by lasso, can we then suggest that that predictor was weak/low variance/irrelevant?

Improve this question
$\endgroup$
3
  • $\begingroup$ Where do you get your first paragraph from? Do you mean variance or co-variance? $\endgroup$
    – Henry
    Commented Aug 18, 2022 at 9:17
  • $\begingroup$ On your second paragraph, I suspect in a hand waving sense the dropped predictors had a higher "cost/benefit ratio" than retained predictors $\endgroup$
    – Henry
    Commented Aug 18, 2022 at 9:20
  • $\begingroup$ normally, predictors are scaled to unit variance. So its more that shrinkage is most extreme in directions of low correlation between predictors.( so its 'averaging' over predictors). $\endgroup$
    – seanv507
    Commented Aug 18, 2022 at 12:44

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.