Timeline for How many zeroes from lasso linear regression?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 24 at 15:21 | comment | added | george1994 | okay, I formulated a question stats.stackexchange.com/questions/649818/…. Please feel free to edit, I am still not that advanced in the field, so tried my best. | |
| Jun 24 at 10:41 | comment | added | Sextus Empiricus | On the other side you could ask, "are there cases in elastic net, different from lasso, where all coefficients are non-zero?", that question has a more simple answer (there are no cases) but I believe that it can still be asked and some answers might explain intuitively the difference between lasso and ridge and what happens when they are mixed. | |
| Jun 24 at 10:37 | comment | added | Sextus Empiricus | @george1994 if you like, you can go ahead. The lasso case is covered in stats.stackexchange.com/questions/289075/… , for elastic net I suggest it would be have the title "What is the boundary curve for $\lambda_1$ and $\lambda_2$ that give at least a 0 component in elastic net?" | |
| Jun 24 at 9:58 | comment | added | george1994 | Do you want to formulate a question together? | |
| Jun 24 at 9:49 | comment | added | Sextus Empiricus | @george1994 To be honest, I never thought about it, but certainly it's gonna be different (and at least slightly more complex). This might actually create an interesting question, the combination of penalty terms that limit between sparse and non-sparse. (although currently this sounds just theoretical, I am not sure yet about the practical use of it) | |
| Jun 24 at 8:55 | comment | added | george1994 | understand, thanks. So practically, when implementing an Elastic Net and defining the grid for the hyperparameters, will it be just based on testing, where a combination leads to a complete sparse and non-sparse solution? | |
| Jun 24 at 8:29 | comment | added | Sextus Empiricus | @george1994 for Elastic Net one can also have sparse solutions but the smallest $\lambda_{\text{all zero}}$ is at infinity. And the smallest $\lambda_{\text{non zero}}$ is not so easily computed and might be at infinity as well (if the solutions are never sparse for any regularisation). And, if instead of using a fixed mixing parameter, you use two regularisation parameters for the two regularisation terms, then it becomes complex to describe as well. | |
| Jun 24 at 7:23 | comment | added | george1994 | Is this also true for the Elastic Net estimation? | |
| Mar 11 at 11:47 | vote | accept | alexmolas | ||
| Mar 11 at 11:47 | history | bounty ended | alexmolas | ||
| Mar 11 at 10:50 | history | edited | Sextus Empiricus | CC BY-SA 4.0 |
added 156 characters in body
|
| Mar 11 at 10:29 | history | edited | Sextus Empiricus | CC BY-SA 4.0 |
added 831 characters in body
|
| Mar 6 at 9:00 | history | edited | Sextus Empiricus | CC BY-SA 4.0 |
added 1142 characters in body
|
| Mar 5 at 10:31 | history | answered | Sextus Empiricus | CC BY-SA 4.0 |