Timeline for When is it valid to use lasso and adaptive lasso
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 26, 2021 at 14:28 | comment | added | EdM |
@user54285 SAS/STAT has implemented LASSO for logistic regression since version 14.1 in the HPGENSELECT procedure. Ridge can help with problems besides multicollinearity: it penalizes coefficients to reduce overfitting and to avoid "complete separation" problems. The R LASSO is nicely illustrated in Section 6.6.2 of ISLR, ridge in Section 6.6.1; although shown for a Gaussian model, you just specify family="binomial" for logistic models.
|
|
| Sep 26, 2021 at 2:08 | comment | added | user54285 | The sas product I am using always builds linear models in generating lasso. Again what I read that commented on this said this was not a big issue if using it just to select variables. But from your comments I don't think you agree. So in the future I am not going to do this. I do not know R well enough to utilize the R modules for lasso. I use it for limited things, mainly time series. I am trying to learn it more - historically I used only SAS. By the way thank you for your comments. | |
| Sep 26, 2021 at 1:59 | comment | added | user54285 | I wanted to point out that I ran a VIF, using linear regression, but I don' this matters for testing multicollinearity, for these variables and no VIF was higher than 3 while the normal warning level is 5, or some say 10. It is true that many of the 39 variables are not statistically significant. My concern was not multicollinearity, which is why I understand you use Ridge, but simply have too many predictors for the data. I thought lasso was preferred to ridge in selecting variables. I am not weeded to adaptive lasso, but the fact that it shrinks the smaller slopes more seemed to make sense. | |
| Sep 26, 2021 at 1:50 | comment | added | user54285 | This came about because I felt we had too many predictors for a logistic regression model (although the model ran). We had 39 predictors and only about 45 cases at one level of the DV (out of about 445 total). From what I have read, e.g., agressti, it is unwise to have more than one predictor for every ten cases. We have no theory at all to throw out some predictors. A researcher I know said for sparse data like this LASSO would be a good way to reduce the list of variables. One article I read suggested that the fact the DV was categorial would not matter for this. But the lasso won't run. | |
| Sep 21, 2021 at 15:46 | history | answered | EdM | CC BY-SA 4.0 |