In applying a penalty term with either $l_1$ or $l_2$ norm, why would the former result in variable selection but not the latter?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ Duplicate of this. Also see here or this diagram $\endgroup$– Glen_bCommented May 2, 2014 at 5:06
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The constrained region for l1 is diamond shaped(pointed at axis) while for l2 it is round(in 2 dimensions, similary rhomboid and sphere in case of higher dimensions). The residual sum of squares have elliptical contours, centered at full least square estimate. Both methods find first point where elliptical contours hit the constrained region. Since rhobus has corners at axis one is more likely to find a solution where one of the coefficients is 0.
Please refer to Elements o Statistical Learning by Tibshirani and Hastie(second edition page 71).
