Whenever someone writes about Lasso and Ridge Regression thy draw this diagram with the circle or with the diamond.
In the case of the diamond (Lasso regression) it is then always stated that Lasso forces one of the coefficients to 0. Therefor it introduces sparsity. I understand it somehow, but whenever I see the diagram my doubts return. Why couldn't one just draw it like this:
Obviously none of the coefficients is forced to zero in this case. Both can take number between -1 and 1. What am I missing? My drawing has to be wrong, but I don't get it why they always draw so that it hits $\beta_1=0$
Just found this quote:
However, the lasso constraint has corners at each of the axes and so the ellipse will often intersect the constraint region at an axis
Is that it? It will intersect often with the constraint region, but it doesn't have to? Can't wrap my head around it. I can only imagine that in higher dimensional cases hitting a corner becomes more likely or even inevitable.