I have a theoretical question. I was reading about ridge regression, lasso and the elastic net, and is very impressed. One thing is not quite clear to me.
I would like to know when should I use each model: Linear regression, ridge, lasso and elastic net. What are the advantages and disadvantages of them all ?
I know that an advantage of the lasso is that it does not only shrinkage but also feature selection, it can force the coefficients to zero. Is there anything else ? Why using ridge then, ever ? And what about elastic net, can it shrink coefficients to zero? why would anyone want to combine lasso with ridge, I mean, lasso also shrinks the coefficients.
I am slightly confused.