I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact is that for Lasso solution $\hat\beta$ subject to $\hat{\pmb{\mu}}={\bf{X}}\hat\beta$, the sign of any non-zero coordinate $\hat\beta_j$ must agree with the sign of the relevant correlation $\hat{c_j}={\bf{x}}_j^T({\bf{y}}-\hat{\pmb{\mu}})$, and LARS doesn't have that restriction. For LARS, the sign of $\hat\beta_j$ may change. The reason is given in the same section in the paper, and it's not too difficult to understand algebraically.

My question is how to understand this fact in a geometric way. As mentioned in that paper, for LARS, at each stage the estimator $\hat{\pmb{\mu}}_k$ approaches, but does not reach, the projection of $\bf{y}$ onto the subspace spanned by the active set. The path makes a bend (adds one dimension) at each stage to a higher-dimension projection. But I cannot imagine the sign of any coefficient $\hat\beta_j$ will change on the way. I failed to construct a 3-dim example.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.