let's consider the lasso problem as: $$ \frac{1}{2}\|y-X\beta\|_2^2 + \lambda\|\beta\|_1 $$ and referring to The lasso by Patrick Breheny enter image description here

he says the correlation between a predictor and residuals, must exceed a certain minimum threshold λ before it is included in the model.

what if we chose λ equals 2 or any value larger than 1. how that statement could be true? as we know the correlation (Pearson correlation) lies between -1,1. or let's say by that statement if the correlation between a predictor and residuals =0.99 and λ = 2 (arbitrary example) then none of the predictors of my model won't be included in the model

  • 3
    $\begingroup$ That's not a correlation, that's a covariance. I wonder if he just experienced a brief mental lapse while typing in his slides? $\endgroup$
    – jbowman
    Dec 27, 2022 at 18:29
  • $\begingroup$ I think it's a correlation, not a covariance as all variables are standardized. $\endgroup$
    – Nidal
    Dec 28, 2022 at 4:30
  • 1
    $\begingroup$ @Nidal a residual is not standardized. $\endgroup$
    – AdamO
    Dec 28, 2022 at 6:01
  • $\begingroup$ @AdamO "the variables that have equal (and maximal) absolute correlation with the residual" $\endgroup$
    – Nidal
    Dec 28, 2022 at 16:16
  • $\begingroup$ stats.stackexchange.com/questions/351552/… read this also $\endgroup$
    – Nidal
    Dec 28, 2022 at 16:22


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.