I have question regarding the shrinking of a coefficient depending on the size of $\lambda$ over the regularization path. Let's assume we have for a given $\lambda$ a lasso solution $$ \hat{\beta}(\lambda )\neq 0.$$
Does it always hold for $\lambda'\leq \lambda$.that then we have $$ \hat{\beta}(\lambda' )\neq 0 $$ as well ?
I know that if we increase $\lambda$ it can happen that the estimated coefficient can inrease as well (at least a bit),e.g if another coefficient is shrunken to exactly zero. This can also be observed of the regularization paths. But I actually never thought about the question asked above.
