18 questions linked to/from Why does shrinkage work?
12k views

### What is penalized logistic regression [duplicate]

I need to do a logistic regression that will likely have a lot of zeros. Can someone explain penalized logistic regression to me like I'm dumb?
2k views

### Why does regularization of coefficient magnitude improve the generalization of linear regression? [duplicate]

What is the basic argument upon which ridge and lasso regression are based on? I went through Tikhonov regularization wiki where it was mentioned that In many cases, tikhonov matrix is chosen as ...
248 views

### Why were 'Regularization' methods (Lasso, Ridge, Elastic Net) created in the first place? [duplicate]

What problem do regularization methods solve? I thought it was feature selection and to prevent overfitting. However, I was informed that the reason Ridge, Lasso, and Elastic Net were created in the ...
217 views

### Linear Regression — Regularization, shrinkage [duplicate]

Regularization reduces the magnitudes of the regression coefficients. I read that this helps reduce the variance of the model. Why exactly do smaller values of the coefficients lead to a lower ...
61 views

### Why is L2 regression good for handling multicollinearity? [duplicate]

Looking for an intuitive explanation, thanks.
34 views

### LASSO method. Intuitively how does it select variables? [duplicate]

Intuitively how does the LASSO method select its variables? Is it based on standard econometrics?
4k views

### How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from dimensionality-reduction/...
4k views

1k views

### Why does Daniel Wilks (2011) say that principal component regression “will be biased”?

In Statistical Methods in the Atmospheric Sciences, Daniel Wilks notes that multiple linear regression can lead to problems if there are very strong intercorrelations among the predictors (3rd edition,...