If we select different values of the parameter $\lambda$, we could obtain solutions with different sparsity levels. Does it mean the regularization path is how to select the coordinate that could get faster convergence? I'm a little confused although I have heard about sparsity often. In addition, could you please give a simple description about the existing solutions of LASSO problem?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
0
Say you have a model with $p$ predictor variables: $x_1, x_2, \ldots x_p$. Set $\lambda$ to an initial value, and estimate your coefficients $\beta_1, \beta_2, \ldots \beta_p$. These coefficients can be thought of as a point in $p$-dimensional space.*
Repeat the procedure for your next value of $\lambda$, and get another set of estimates. These form another point in $p$-dimensional space. Do this for all your $\lambda$ values, and you will get a sequence of such points. This sequence is the regularization path.
* There's also the intercept term $\beta_0$ so all this technically takes place in $(p+1)$-dimensional space, but never mind that. Anyway most elastic net/lasso programs will standardise the variables before fitting the model, so $\beta_0$ will always be 0.
