Skip to main content

# Questions tagged [lars]

LARS stands for Least Angle Regression. It is a penalized estimation and feature selection technique for multiple regression.

50 questions
Filter by
Sorted by
Tagged with
0 votes
0 answers
36 views

### Restricting the order in which variables are allowed to enter in the LARS algorithm

In the Least Angle Regression paper, in section 3, the authors refer to how you can restrict the order in which variables are allowed to enter the LARS Algorithm. In particular, having obtained some ...
1 vote
1 answer
220 views

### When modified LARS implements LASSO, what's the corresponding lambda

We know that the modified LARS is used to implement LASSO. However what's the corresponding lambda? As there is no lambda parameter in LARS. I found reference here: LASSO regularisation parameter from ...
• 1,345
0 votes
0 answers
156 views

### LARS package in R with specific lambda value

I am trying to use the LARS package in R to obtain a Lasso estimate of the sparse coefficient vector, say $\hat{\beta}_{\text{sparse}}$, as opposed to a coefficient path. In other words, I am not ...
• 1,226
0 votes
0 answers
202 views

### glmnet vs. lars for computing the lasso

I've seen this post as well as this one regarding the difference between the lars and glmnet solution paths for fitting the ...
• 253
0 votes
0 answers
49 views

### Model fit after inverse regression and forward stagewise on residuals

I work with prediction of height based on genetic information. The data is several hundred thousand data points from every organism, but a simplified version could be: ...
0 votes
1 answer
329 views

### What does lars return for lambda equal to zero when p is larger than n?

For the lasso problem $$\hat \beta(\lambda) = \underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}}\|Y - X \beta\|_2^2 + \lambda \|\beta\|_1,$$ where $Y$ is $p \times 1$, $X$ is $n \times p$, it ...
0 votes
1 answer
117 views

### alpha and cv_alpha parameters in sklearn.linear_model.LarsCV

Can someone explain alpha and cv_alpha parameters in sklearn.linear_model.LarsCV? I am guessing that alphas refer to maximum correlation at any given step between one of the remaining explanatory ...
• 1
0 votes
2 answers
1k views

### using the Lasso in sklearn [closed]

I am a beginner, and I am trying to use the Lasso to do some regression. I am looking specifically at the LassoLars module in sklearn. What I am really after is recovering the parameter weight vector ...
1 vote
0 answers
30 views

### Cross-validated methods as a default

Scikit-learn ( http://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model ) provide classes for Lasso, ElaticNet and Lars etc both with and without cross-validation. What are ...
• 10.1k
0 votes
0 answers
369 views

### Drawbacks of Lasso-lars combined with recursive feature elimination

During the model selection phase of my (regression) work I noticed that Lars with lasso modification performs way worse than any other model if combined with recursive feature elimination, both in ...
• 177
1 vote
0 answers
267 views

### Stacking Lasso models

let say I have M subsets of independent variables and I want to use stack learner to predict dependent variable y. for each subset I use lasso method to get meta features (predictions). I have 2 ...
• 111
2 votes
1 answer
724 views

### How to get the equiangular vector in p dimension linear space? Used in Least angle regression

Here I am reading the "Least angle regression" by Efron, 2004. https://projecteuclid.org/download/pdfview_1/euclid.aos/1083178935 , in page 413, he gives formula of equiangular vector of the linear ...
• 23
2 votes
0 answers
117 views

### How can lasso CCA be solved using LARS?

According to paper By Sun, Ji an Ye; A Least Squares Formulation for Canonical Correlation Analysis http://www.machinelearning.org/archive/icml2008/papers/270.pdf CCA can be reformulated as a least ...
• 7,709
5 votes
1 answer
661 views

### What is the rationale behind LARS-OLS hybrid, i.e. using OLS estimate on the variables chosen by LARS?

I need some help to understand the relationship between the ranking of the variables from the LARS algorithm and the use of OLS to estimate the final model chosen by the LARS. I understand that the ...
• 51
4 votes
1 answer
674 views

### Classification with Least Angle (LARS)-type algorithm?

I am currently working on the LARS (Least Angle Regression) method. I know it is a regression method, but I wonder if, like LASSO or Ridge techniques (e.g. the package gmlnet in R), it can be ...
• 41
2 votes
0 answers
169 views

### question in least angle regression paper [Efron et al. (2004)]

I am reading Efron et al. (2004), Ann. Statist., 32, 2, "Least angle regression". I have struggled with formula 5.19 for some days. It says the minimizing resolution of lasso occurs strictly inside ...
0 votes
0 answers
694 views

### Inconsistent results for LASSO (glmnet package)

I am getting inconsistent results (coefficients and the mean cross-validated error) with the glmnet package in R. The data set has 33 variables and 250 observations. This is the sample output for the ...
• 251
5 votes
1 answer
1k views

### Why under joint least squares direction is it possible for some coefficients to decrease in LARS regression? [duplicate]

I think I understand how LARS regression works. It basically adds features to the model when they are more correlated with the residuals than the current model. ...
• 2,289
1 vote
1 answer
2k views

### Cross-validation and LASSO plot

I am not sure about some parts of the following plots: -- I do not understand what does the expression on x-axis means (first figure) and how to interpret that. -- What is the bars and the points in ...
• 147
3 votes
1 answer
2k views

### LARS vs LASSO and Cross-validation

I would like to apply lars algorithm to some datadset. First, I fitted the model to the training set and then examined it on test set. My questions: 1- After I used cross validation "cv.lars" I ...
• 147
2 votes
3 answers
2k views

### What is "step size" in LASSO regression?

I'm looking at this example for LASSO regression in R: http://machinelearningmastery.com/penalized-regression-in-r/. It ...
• 2,289
1 vote
1 answer
297 views

### Cross-validated prediction error worse for LARS than ordinary linear regression

I am analysing microarray data in order to build a model for predicting cell proliferation (a continuous variable) based on gene expression (also a continuous variable). There are many more genes than ...
4 votes
1 answer
1k views

### Computational complexity of the lasso (lars vs coordinate descent)

The lasso can be computed with the LARS or Coordinate Descent algorithm. What is their computational complexity and when one is quicker than the other?
• 3,179
10 votes
1 answer
2k views

### LASSO regularisation parameter from LARS algorithm

In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths. I have implemented this ...
• 1,556
1 vote
0 answers
166 views

### How to imagine (visualize) the difference between LARS and Lasso

I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact ...
• 91
2 votes
0 answers
300 views

• 691
45 votes
5 answers
121k views

### Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
• 451
2 votes
1 answer
1k views

### Least angle regression packages for R or MATLAB

I am looking for a Least Angle Regression (LAR) packages in R or MATLAB which can be used for classification problems. The only package that I currently know which fits this description is glmpath. ...
• 5,075
1 vote
1 answer
734 views

### 'Forward Stagewise' option in LARS algorithm

Can anyone help me understand the forward stagewise part in the LARS algorithm? I was reading the R code and could not figure out what is ...
• 13
0 votes
1 answer
287 views

### Max steps in lars

I don't know how many steps are necessary for lars() to select the variables till the algorithm proceeds to the saturated fit (especially using the ...
• 19
3 votes
0 answers
107 views

### Constant signs of correlation in the active set in least angle regression

I am trying to comprehend the proof of the Least Angle Regression algorithm and I am stuck at certain points. I would appreciate any help that I can get. Let me set the stage: I am following the ...
• 391
8 votes
1 answer
3k views

### LARS - LASSO with weights

I am interested in solving the following problem  \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + \...
• 391
30 votes
2 answers
5k views

### Advantages of doing "double lasso" or performing lasso twice?

I once heard a method of using the lasso twice (like a double-lasso) where you perform lasso on the original set of variables, say S1, obtain a sparse set called S2, and then perform lasso again on ...
• 871
15 votes
2 answers
992 views

### LASSO/LARS vs general to specific (GETS) method

I have been wondering, why are LASSO and LARS model selection methods so popular even though they are basically just variations of step-wise forward selection (and thus suffer from path dependency)? ...
• 151
71 votes
5 answers
17k views

### What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
• 14.2k
6 votes
1 answer
737 views

### Feature selection with k-fold cross-validated least angle regression

I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$). I have seven predictors ($x_1,x_2,...,x_7$) for each ...
• 1,989