16k views

### Sparsity in Lasso and advantage over ridge (Statistical Learning) [duplicate]

I'm learning about the Statistical learning and in the section comparing Lasso and Ridge Regression it shows that the main difference between these two problems is the way the constraint/penalty is ...
• 8,837
4k views

### What is the mathematical rigorous proof that L1 regularization will give sparse solution? [duplicate]

It is given in the book Machine Learning A probabilistic Perspective, but i am not able to understand it. Can some one provide an explanation for that ? I am not clear with the way sub gradient is ...
10k views

### Why L1 regularization can "zero out the weights" and therefore leads to sparse models? [duplicate]

I'm aware there is a very relevant explanation on L1 regularization's effect on feature selection at here: Why L1 norm for sparse models [Ref. 1]. To better understand it I'm reading Google's ...
• 2,365
1k views

### Difference between L1 and L2 Regularization (in Lasso and Ridge Regression) [duplicate]

I got a more theoretical question here: I have done some research about the L2 (Ridge) and L1 (Lasso) regularizations. I know the formula, and understand the aim of those two different procedures. The ...
1 vote
2k views

### why l1 norm can result in variable selection but not l2 [duplicate]

In applying a penalty term with either $l_1$ or $l_2$ norm, why would the former result in variable selection but not the latter?
• 1,285
707 views

### Why feature selection using L1 and not using L2 norm? [duplicate]

I read a tutorial here. In which, I came across the below plots I read an explanation quoted ...
• 3,008
1 vote
280 views

### Why does not ridge regression perform feature selection although it makes use of regularization? [duplicate]

Could somebody explain why ridge regression does not perform feature selection although it makes use of regularization? So, it penalizes the regression coefficients like LASSO does, but how come we ...
• 1,509
193 views

### Make weights zero in L1 regularization [duplicate]

L1 regularization reduces the magnitude of weights but does not make them zero. Is there a way we can make them zero. Will it be better than normal L1 regularization
167 views

### LASSO method. Intuitively how does it select variables? [duplicate]

Intuitively how does the LASSO method select its variables? Is it based on standard econometrics?
1 vote
40 views

### Does L1 regularization (Lasso) always leads to feature reduction? [duplicate]

This is a basic question about regularization term but I have searched for a while and cannot find the answer. My question is: does Lasso regularization always make some coefficients zero? A famous ...
• 145
33 views

### How does Ridge regression / Regularization help in selecting less or more important features? [duplicate]

Can someone please explain how regularization helps to shrink the " less important " features to zero ? As far as I know , Regularization only penalizes the weights of ALL the features to ...
• 337
1 vote
18 views

### Connection Between Bayesian Prior and Variable Selection in Lasso [duplicate]

I am interested in learning more about the Bayesian interpretation of the Lasso model. The Lasso model assumes a Laplace distribution of coefficients and the optimal coefficients maximize the ...
• 367
15 views

### Why does Lasso Regression only eliminate some features [duplicate]

How does Lasso Regression select which coefficients to set to 0 and why are not all of them set to zero? My understanding is minimizing the function:  min_{\beta} \lvert\lvert y-X\beta\rvert\rvert^{...
• 111
86k views

### What is the lasso in regression analysis?

I'm looking for a non-technical definition of the lasso and what it is used for.
• 881
31k views

### Why will ridge regression not shrink some coefficients to zero like lasso?

When explaining LASSO regression, the diagram of a diamond and circle is often used. It is said that because the shape of the constraint in LASSO is a diamond, the least squares solution obtained ...

15 30 50 per page