Conceptually, I don't understand what least angle regression Least Angle Regression (LARS) is and why it solves LASSO (pdf).

We know that LASSO is:

$$\arg \min_x {\left\| A x - y \right\|}_{2}^{2} + \lambda {\left\| x \right\|}_{1}$$

From my understanding LARS is exactly like line search from CGD, where you take the variable that gives you the most desire result and iterate through.

Can someone give a walk through on how LARS solves LASSO?

  • $\begingroup$ An edit request suggests that the LASSO be written as $\arg \min_x {\left\| A x - y \right\|}_{2}^{2} + \lambda {\left\| x \right\|}_{1}$ instead. But this should really be a comment rather than an edit, since fixing errors in someone else's equations is not really a job for the editing system $\endgroup$
    – Silverfish
    Commented Nov 21, 2015 at 12:58
  • $\begingroup$ Why not? I'm interested in that question. It was formalized wrong the other time. Instead of opening new one, I fixed the errors here and hopefully someone would answer. There is no question here, the LASSO Problem is well defined. Moreover, it seems to be abandoned by the OP (He doesn't seem to be active anymore). $\endgroup$
    – Royi
    Commented Nov 21, 2015 at 13:06
  • $\begingroup$ @Drazick It might be good to take that up in the meta thread Silverfish pointed to, perhaps in a comment to an answer you disagree with $\endgroup$
    – Glen_b
    Commented Nov 21, 2015 at 14:55
  • 1
    $\begingroup$ have you read section 3.4.4 of ESL II? $\endgroup$ Commented Feb 27, 2018 at 6:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.