# What is the meaning of regularization path in LASSO or related sparsity problems?

If we select different values of the parameter $\lambda$, we could obtain solutions with different sparsity levels. Does it mean the regularization path is how to select the coordinate that could get faster convergence? I'm a little confused although I have heard about sparsity often. In addition, could you please give a simple description about the existing solutions of LASSO problem?

Say you have a model with $p$ predictor variables: $x_1, x_2, \ldots x_p$. Set $\lambda$ to an initial value, and estimate your coefficients $\beta_1, \beta_2, \ldots \beta_p$. These coefficients can be thought of as a point in $p$-dimensional space.*
Repeat the procedure for your next value of $\lambda$, and get another set of estimates. These form another point in $p$-dimensional space. Do this for all your $\lambda$ values, and you will get a sequence of such points. This sequence is the regularization path.
* There's also the intercept term $\beta_0$ so all this technically takes place in $(p+1)$-dimensional space, but never mind that. Anyway most elastic net/lasso programs will standardise the variables before fitting the model, so $\beta_0$ will always be 0.
• Thank you,sir! Yes! The Fig. 3.10 is the lasso regularization path. – mining Aug 21 '14 at 22:15