Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Least-angle regression and the lasso tend to produce very similar regularization paths (identical except when a coefficient crosses zero.)
They both can be efficiently fit by virtually identical algorithms.
Is there ever any practical reason to prefer one method over the other?
The "no free lunch" theorems suggest that there are no a-priori distinctions between statistical inference algorithms, i.e. whether LARS or LASSO works best depends on the nature of the particular dataset. In practice then, it is best to try both and use some reliable estimator of generalisation performance to decide which to use in operation (or use an ensemble). As the differences between LARS and LASSO are rather slight, the differences in performance are likely to be rather slight as well, but in general there is only one way to find out for sure!
When used in stage-wise mode, the LARS algorithm is a greedy method that does not yield a provably consistent estimator (in other words, it does not converge to a stable result when you increase the number of samples).
Conversely, the LASSO (and thus the LARS algorithm when used in LASSO mode) solves a convex data fitting problem. In particular, this problem (the L1 penalized linear estimator) has plenty of nice proved properties (consistency, sparsistency).
I would thus try to always use the LARS in LASSO mode (or use another solver for LASSO), unless you have very good reasons to prefer stage-wise.
As mentioned before, LARS is a particular method to solve the Lasso problem, i.e. the $l_1$-regularized least squares problem. Its success stems from the fact that it requires an asymptotic effort comparable to standard least-squares regression, and thus a highly superior performance than required by the solution of a quadratic programming problem. Later extensions of LARS also adressed the more general elastic-net problem where you include a sum of $l_1$ and $l_2$-regularization terms into the least-squares functional.
The intention of this answer is to point out that LARS nowadays seems to have been superseeded by coordinate-descent and stochastic coordinate-descent methods. These methods are based on particularly simple algorithms, while at the same time the performance seems to be higher than that of LARS (often one or two orders of magnitude faster). For examples see this paper of Friedman et al.
So, if you plan to implement LARS, don't. Use coordinate-descent which takes a few hours.
LASSO is not an algorithm per se, but an operator.
There are many different ways to derive efficient algorithms for $\ell_1$ regularized problems. For instance, one can use quadratic programming to them tackle directly. I guess this is what you refer to as LASSO.
Another one is LARS, very popular because of its simplicity, connection with forward procedures (yet not too greedy), very constructive proof and easy generalization.
Even compared with state of the art quadratic programming solvers, LARS can be much more efficient.
The computation of the lasso solutions is a quadratic programming problem, and can be tackled by standard numerical analysis algorithms. But the least angle regression procedure is a better approach. This algorithm exploits the special structure of the lasso problem, and provides an efficient way to compute the solutions simultaneously for all values of $\lambda$.
Here is my opinion:
Your question can be divided to two parts. High dimensional cases and low dimensional cases. On the other hand it depends on what criteria are you going to use for selecting the optimal model. in the original paper of LARS, it is proved a $C_p$ criteria for selecting the best model and also you can see a SVS and CV criteria in the 'Discussion' of the paper as well. Generally, there are tiny differences between LARS and Lasso and can be ignored completely.
In addition LARS is computationally fast and reliable. Lasso is fast but there is a tiny difference between algorithm that causes the LARS win the speed challenge. On the other hand there are alternative packages for example in R, called 'glmnet' that work more reliable than lars package(because it is more general).
To sum up, there is nothing significant that can be considered about lars and lasso. It depended on the context you are going to use model.
I personally advise using glmnet in R in both high and low dimensional cases. or if you are interested in different criteria, you can use http://cran.r-project.org/web/packages/msgps/ package.
In some contexts a regularized version of the least squares solution may be preferable. The LASSO (least absolute shrinkage and selection operator) algorithm, for example, finds a least-squares solution with the constraint that | β | 1, the L1-norm of the parameter vector, is no greater than a given value. Equivalently, it may solve an unconstrained minimization of the least-squares penalty with α | β | 1 added, where α is a constant (this is the Lagrangian form of the constrained problem.) This problem may be solved using quadratic programming or more general convex optimization methods, as well as by specific algorithms such as the least angle regression algorithm. The L1-regularized formulation is useful in some contexts due to its tendency to prefer solutions with fewer nonzero parameter values, effectively reducing the number of variables upon which the given solution is dependent.[11] For this reason, the LASSO and its variants are fundamental to the field of compressed sensing.
