1
$\begingroup$

I'm trying to understand the LASSO-path algorithm that is described here on page 9&10 (Sparsity and the Lasso by Tibshirani & Wasserman, 2015).

First they say we initialize $\lambda_0 := \infty$,$s_A = A = \emptyset$. But I don't understand why we set $A = \emptyset$ as all the following computations involve $X_A$ which denotes the columns of the matrix $X$ whose indices are in $A$, which does not make sense if $A = \emptyset$.

Can anyone point out what I'm missing? / How this algorithm starts?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The lasso (typically) does not penalize the intercept, and as the penalization parameter goes to infinity, the lasso model is an intercept-only model.

This is consistent with $X_\emptyset$ being a matrix that only contains this intercept. The algorithm starts with an intercept-only overall mean model. If you go through the other steps of the algorithm, you end up with the classical regularization paths of the parameter estimates, as in Figure 2 in the PDF, coming in from the right.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This specific version does not include a intercept parameter, but I just found the description below the figure you mentioned, where they describe how to choose $\lambda_1$, so that should work. $\endgroup$
    – flawr
    Commented Nov 21, 2017 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.