LARS stands for Least Angle Regression. It is a feature selection technique for multiple regression that incorporates a penalty.

learn more… | top users | synonyms

6
votes
1answer
108 views

LASSO regularisation parameter from LARS algorithm

In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths. I have implemented this ...
1
vote
0answers
44 views

How to imagine (visualize) the difference between LARS and Lasso

I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact ...
1
vote
0answers
44 views

Convergence analysis for forward stagewise regression?

Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor ...
3
votes
0answers
77 views

R - CV.GLMNET LASSO (binomial) variable reduction - highly correlated variables not zeroing out

I have a dataset of about n = 100,000 observations and p = 247 predictors with one binomial dependent variable (values are 0, 1) I run the following code in R: ...
0
votes
1answer
59 views

lasso - how to evaluate results

I'm working on lasso as an alternative to step-wise forward/backward regression using the lars package in R. I normalized my variables, calculated the ...
0
votes
0answers
19 views

explaining LARS algorithm in a simple way [duplicate]

I am thinking to write a LARS algorithm that uses different optimizations in each step. Can somebody briefly explain Least angel regression, LARS (see here) to me? then I will try to write my own LARS ...
0
votes
0answers
53 views

Prediction performance of OLS and Lasso

I am running a comparison of prediction performance of two model using OLS and LASSO respectively. LASSO estimates are computed from LARS algorithm, AIC and BIC were used in model selection. In ...
0
votes
1answer
260 views

How to interprete lasso from lars correctly?

I tried the lars package with R and got the following result. ...
0
votes
0answers
34 views

combining two sources of knowledge about covariates in logistic regression model

i have a number of variables and a dataset and i want to build a linear regression model with shrinkage like lasso. i have also another information about my covariates on their relation with ...
0
votes
0answers
18 views

doing some modifications in LARS method

Lars method uses 'correlation' in order to select and enter variables into the regression model. in my research work, i am trying to use some other parameters in order to control the variables ...
0
votes
0answers
68 views

LASSO / lars-package / variable ranking

I would like to write a generic R-code, which produces a vector with variable rankings according to the optimal LASSO-sequence based on Cp-Criterion. Please see my simple example below. ...
1
vote
0answers
86 views

how to combine coefficients of a logistic regression model with existing prior knowledge about covariates?

I am working on developing statistical models for fault-localization. on the one hand, i construct a logistic regression model with these considerations: 1-my dependent(response) variable is program ...
4
votes
1answer
2k views

R - Lasso Regression - different Lambda per regressor

I want to do the following: 1) OLS regression (no penalization term) to get beta coefficients $b_{j}^{*}$; $j$ stands for the variables used to regress. I do this by ...
4
votes
1answer
809 views

Why is there no intercept in the lars output for LASSO in Stata?

This is my first time using lars, so this question is probably obvious. When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept. I ...
2
votes
1answer
1k views

Deviance measure in glmnet package

for my current reseach I'm using the Lasso method via the glmnet package in R on a binomial dependent variable. In glmnet the optimal lambda is found via cross-validation and the resulting models ...
2
votes
2answers
433 views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
1
vote
0answers
75 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
2
votes
1answer
71 views

Dataset for Least Angle Regression

I have read that least angle regression is good for high dimensional data. I didn't actually understand the meaning of high dimensional data, so does this mean $p>>n$ case? And does anyone know ...
4
votes
0answers
121 views

What is least angle regression?

Conceptually, I don't understand what least angle regression Least Angle Regression (LARS) is and why it solves LASSO (pdf). We know that LASSO is: $$\arg \min_x {\left\| A x - y \right\|}_{2}^{2} ...
1
vote
0answers
415 views

Parameter tuning in lars (lasso) matlab

I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm). I want to do a leave one out cross validation on my data using this code. I have the following ...
0
votes
0answers
125 views

How to add a non-negative constraint to lasso4j?

Lasso4j is a Java implementation of the Lasso L1-constrained fitting for linear regression. I would like to add a non-negativity constraint on the weights, meaning that the non-zero sparse ...
2
votes
3answers
197 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in ...
14
votes
5answers
26k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
1answer
574 views

Least angle regression packages for R or MATLAB

I am looking for a Least Angle Regression (LAR) packages in R or MATLAB which can be used for classification problems. The only package that I currently know which fits this description is glmpath. ...
1
vote
1answer
244 views

'Forward Stagewise' option in LARS algorithm

Can anyone help me understand the forward stagewise part in the LARS algorithm? I was reading the R code and could not figure out what is ...
0
votes
1answer
132 views

Max steps in lars

I don't know how many steps are necessary for lars() to select the variables till the algorithm proceeds to the saturated fit (especially using the ...
2
votes
0answers
71 views

Constant signs of correlation in the active set in least angle regression

I am trying to comprehend the proof of the Least Angle Regression algorithm and I am stuck at certain points. I would appreciate any help that I can get. Let me set the stage: I am following the ...
4
votes
1answer
636 views

LARS - LASSO with weights

I am interested in solving the following problem $$ \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + ...
17
votes
2answers
868 views

Advantages of doing “double lasso” or performing lasso twice?

I once heard a method of using the lasso twice (like a double-lasso) where you perform lasso on the original set of variables, say S1, obtain a sparse set called S2, and then perform lasso again on ...
13
votes
1answer
376 views

LASSO/LARS vs general to specific (GETS) method

I have been wondering, why are LASSO and LARS model selection methods so popular even though they are basically just variations of step-wise forward selection (and thus suffer from path dependency)? ...
45
votes
4answers
7k views

What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
4
votes
1answer
448 views

Feature selection with k-fold cross-validated least angle regression

I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$). I have seven predictors ($x_1,x_2,...,x_7$) for each ...