LARS stands for Least Angle Regression. It is a feature selection technique for multiple regression that incorporates a penalty.

learn more… | top users | synonyms

0
votes
1answer
34 views

What is “step size” in LASSO regression?

I'm looking at this example for LASSO regression in R: http://machinelearningmastery.com/penalized-regression-in-r/. It ...
0
votes
0answers
14 views

Can LARS or Coordinate Descent select features that are marginally uncorrelated with the response?

I could construct a response Y the following way: Given $\left\lbrace X_k \right\rbrace_{k=1}^p$, and the regression model $$ Y = \sum_{i = 1}^p X_i - \rho p \beta_{p+1}X_{p+1} + \varepsilon,$$ if ...
1
vote
1answer
45 views

Cross-validated prediction error worse for LARS than ordinary linear regression

I am analysing microarray data in order to build a model for predicting cell proliferation (a continuous variable) based on gene expression (also a continuous variable). There are many more genes than ...
1
vote
0answers
13 views

Computational complexity of the lasso (lars vs coordinate descent)

The lasso can be computed with the LARS or Coordinate Descent algorithm. What is their computational complexity and when one is quicker than the other?
6
votes
1answer
151 views

LASSO regularisation parameter from LARS algorithm

In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths. I have implemented this ...
1
vote
0answers
50 views

How to imagine (visualize) the difference between LARS and Lasso

I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact ...
1
vote
0answers
49 views

Convergence analysis for forward stagewise regression?

Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor $\hat{...
3
votes
0answers
91 views

R - CV.GLMNET LASSO (binomial) variable reduction - highly correlated variables not zeroing out

I have a dataset of about n = 100,000 observations and p = 247 predictors with one binomial dependent variable (values are 0, 1) I run the following code in R: ...
0
votes
1answer
67 views

lasso - how to evaluate results

I'm working on lasso as an alternative to step-wise forward/backward regression using the lars package in R. I normalized my variables, calculated the ...
0
votes
0answers
20 views

explaining LARS algorithm in a simple way [duplicate]

I am thinking to write a LARS algorithm that uses different optimizations in each step. Can somebody briefly explain Least angel regression, LARS (see here) to me? then I will try to write my own LARS ...
0
votes
0answers
60 views

Prediction performance of OLS and Lasso

I am running a comparison of prediction performance of two model using OLS and LASSO respectively. LASSO estimates are computed from LARS algorithm, AIC and BIC were used in model selection. In ...
0
votes
1answer
406 views

How to interprete lasso from lars correctly?

I tried the lars package with R and got the following result. ...
1
vote
0answers
90 views

how to combine coefficients of a logistic regression model with existing prior knowledge about covariates?

I am working on developing statistical models for fault-localization. on the one hand, i construct a logistic regression model with these considerations: 1-my dependent(response) variable is program ...
4
votes
1answer
2k views

R - Lasso Regression - different Lambda per regressor

I want to do the following: 1) OLS regression (no penalization term) to get beta coefficients $b_{j}^{*}$; $j$ stands for the variables used to regress. I do this by ...
4
votes
1answer
945 views

Why is there no intercept in the lars output for LASSO in Stata?

This is my first time using lars, so this question is probably obvious. When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept. I ...
2
votes
1answer
1k views

Deviance measure in glmnet package

for my current reseach I'm using the Lasso method via the glmnet package in R on a binomial dependent variable. In glmnet the optimal lambda is found via cross-validation and the resulting models ...
2
votes
2answers
465 views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
1
vote
0answers
84 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ \...
2
votes
1answer
74 views

Dataset for Least Angle Regression

I have read that least angle regression is good for high dimensional data. I didn't actually understand the meaning of high dimensional data, so does this mean $p>>n$ case? And does anyone know ...
4
votes
0answers
128 views

What is least angle regression?

Conceptually, I don't understand what least angle regression Least Angle Regression (LARS) is and why it solves LASSO (pdf). We know that LASSO is: $$\arg \min_x {\left\| A x - y \right\|}_{2}^{2} +...
1
vote
0answers
441 views

Parameter tuning in lars (lasso) matlab

I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm). I want to do a leave one out cross validation on my data using this code. I have the following ...
0
votes
0answers
137 views

How to add a non-negative constraint to lasso4j?

Lasso4j is a Java implementation of the Lasso L1-constrained fitting for linear regression. I would like to add a non-negativity constraint on the weights, meaning that the non-zero sparse ...
2
votes
3answers
220 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in \{1,...,1000\}\}...
17
votes
5answers
30k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
1answer
595 views

Least angle regression packages for R or MATLAB

I am looking for a Least Angle Regression (LAR) packages in R or MATLAB which can be used for classification problems. The only package that I currently know which fits this description is glmpath. ...
1
vote
1answer
251 views

'Forward Stagewise' option in LARS algorithm

Can anyone help me understand the forward stagewise part in the LARS algorithm? I was reading the R code and could not figure out what is ...
0
votes
1answer
133 views

Max steps in lars

I don't know how many steps are necessary for lars() to select the variables till the algorithm proceeds to the saturated fit (especially using the ...
2
votes
0answers
71 views

Constant signs of correlation in the active set in least angle regression

I am trying to comprehend the proof of the Least Angle Regression algorithm and I am stuck at certain points. I would appreciate any help that I can get. Let me set the stage: I am following the ...
4
votes
1answer
665 views

LARS - LASSO with weights

I am interested in solving the following problem $$ \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + \...
18
votes
2answers
935 views

Advantages of doing “double lasso” or performing lasso twice?

I once heard a method of using the lasso twice (like a double-lasso) where you perform lasso on the original set of variables, say S1, obtain a sparse set called S2, and then perform lasso again on ...
14
votes
2answers
417 views

LASSO/LARS vs general to specific (GETS) method

I have been wondering, why are LASSO and LARS model selection methods so popular even though they are basically just variations of step-wise forward selection (and thus suffer from path dependency)? ...
46
votes
4answers
7k views

What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
4
votes
1answer
454 views

Feature selection with k-fold cross-validated least angle regression

I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$). I have seven predictors ($x_1,x_2,...,x_7$) for each ...