Linked Questions
31 questions linked to/from Why does the Lasso provide Variable Selection?
6
votes
1
answer
16k
views
Sparsity in Lasso and advantage over ridge (Statistical Learning) [duplicate]
I'm learning about the Statistical learning and in the section comparing Lasso and Ridge Regression it shows that the main difference between these two problems is the way the constraint/penalty is ...
- 8,837
5
votes
1
answer
4k
views
What is the mathematical rigorous proof that L1 regularization will give sparse solution? [duplicate]
It is given in the book Machine Learning A probabilistic Perspective, but i am not able to understand it. Can some one provide an explanation for that ?
I am not clear with the way sub gradient is ...
- 719
12
votes
1
answer
10k
views
Why L1 regularization can "zero out the weights" and therefore leads to sparse models? [duplicate]
I'm aware there is a very relevant explanation on L1 regularization's effect on feature selection at here: Why L1 norm for sparse models [Ref. 1].
To better understand it I'm reading Google's ...
- 2,365
3
votes
2
answers
1k
views
Difference between L1 and L2 Regularization (in Lasso and Ridge Regression) [duplicate]
I got a more theoretical question here: I have done some research about the L2 (Ridge) and L1 (Lasso) regularizations. I know the formula, and understand the aim of those two different procedures. The ...
- 31
1
vote
1
answer
2k
views
why l1 norm can result in variable selection but not l2 [duplicate]
In applying a penalty term with either $l_1$ or $l_2$ norm, why would the former result in variable selection but not the latter?
- 1,285
0
votes
1
answer
707
views
Why feature selection using `L1` and not using `L2` norm? [duplicate]
I read a tutorial here. In which, I came across the below plots
I read an explanation quoted ...
- 3,008
1
vote
0
answers
280
views
Why does not ridge regression perform feature selection although it makes use of regularization? [duplicate]
Could somebody explain why ridge regression does not perform feature selection although it makes use of regularization? So, it penalizes the regression coefficients like LASSO does, but how come we ...
- 1,509
0
votes
0
answers
193
views
Make weights zero in L1 regularization [duplicate]
L1 regularization reduces the magnitude of weights but does not make them zero. Is there a way we can make them zero. Will it be better than normal L1 regularization
- 111
0
votes
0
answers
167
views
LASSO method. Intuitively how does it select variables? [duplicate]
Intuitively how does the LASSO method select its variables?
Is it based on standard econometrics?
- 149
1
vote
0
answers
40
views
Does L1 regularization (Lasso) always leads to feature reduction? [duplicate]
This is a basic question about regularization term but I have searched for a while and cannot find the answer. My question is: does Lasso regularization always make some coefficients zero?
A famous ...
- 145
0
votes
0
answers
33
views
How does Ridge regression / Regularization help in selecting less or more important features? [duplicate]
Can someone please explain how regularization helps to shrink the " less important " features to zero ? As far as I know , Regularization only penalizes the weights of ALL the features to ...
- 337
1
vote
0
answers
18
views
Connection Between Bayesian Prior and Variable Selection in Lasso [duplicate]
I am interested in learning more about the Bayesian interpretation of the Lasso model. The Lasso model assumes a Laplace distribution of coefficients and the optimal coefficients maximize the ...
- 367
0
votes
0
answers
15
views
Why does Lasso Regression only eliminate some features [duplicate]
How does Lasso Regression select which coefficients to set to 0 and why are not all of them set to zero? My understanding is minimizing the function:
$$ min_{\beta} \lvert\lvert y-X\beta\rvert\rvert^{...
- 111
88
votes
3
answers
86k
views
What is the lasso in regression analysis?
I'm looking for a non-technical definition of the lasso and what it is used for.
- 881
32
votes
2
answers
31k
views
Why will ridge regression not shrink some coefficients to zero like lasso?
When explaining LASSO regression, the diagram of a diamond and circle is often used. It is said that because the shape of the constraint in LASSO is a diamond, the least squares solution obtained ...
