Questions tagged [lars]

LARS stands for Least Angle Regression. It is a penalized estimation and feature selection technique for multiple regression.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Lars alphas_ results

Could any one explain how the results on alphas_ attribute in Lars model are calculated? In the definition: alphas_ is the maximum covariance (abs value) in each iteration. But when I look into ...
0
votes
1answer
15 views

alpha and cv_alpha parameters in sklearn.linear_model.LarsCV

Can someone explain alpha and cv_alpha parameters in sklearn.linear_model.LarsCV? I am guessing that alphas refer to maximum correlation at any given step between one of the remaining explanatory ...
0
votes
0answers
32 views

which parameter you choose on lasso CV, tuning parameter λ or βi constraint s?

I try to use lasso for prediction and I have $X_{tr} \subset X$ the train set and $Y_{tr}$ the train target. and I have $X_{ts} \subset X$ and $ Y_{ts}$ the test set for CV. I used CV and got $λ_i$ ...
0
votes
2answers
254 views

using the Lasso in sklearn [closed]

I am a beginner, and I am trying to use the Lasso to do some regression. I am looking specifically at the LassoLars module in sklearn. What I am really after is recovering the parameter weight vector ...
1
vote
0answers
21 views

Cross-validated methods as a default

Scikit-learn ( http://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model ) provide classes for Lasso, ElaticNet and Lars etc both with and without cross-validation. What are ...
0
votes
0answers
148 views

Drawbacks of Lasso-lars combined with recursive feature elimination

During the model selection phase of my (regression) work I noticed that Lars with lasso modification performs way worse than any other model if combined with recursive feature elimination, both in ...
1
vote
0answers
76 views

Stacking Lasso models

let say I have M subsets of independent variables and I want to use stack learner to predict dependent variable y. for each subset I use lasso method to get meta features (predictions). I have 2 ...
2
votes
1answer
246 views

How to get the equiangular vector in p dimension linear space? Used in Least angle regression

Here I am reading the "Least angle regression" by Efron, 2004. https://projecteuclid.org/download/pdfview_1/euclid.aos/1083178935 , in page 413, he gives formula of equiangular vector of the linear ...
2
votes
0answers
63 views

How can lasso CCA be solved using LARS?

According to paper By Sun, Ji an Ye; A Least Squares Formulation for Canonical Correlation Analysis http://www.machinelearning.org/archive/icml2008/papers/270.pdf CCA can be reformulated as a least ...
4
votes
1answer
261 views

What is the rationale behind LARS-OLS hybrid, i.e. using OLS estimate on the variables chosen by LARS?

I need some help to understand the relationship between the ranking of the variables from the LARS algorithm and the use of OLS to estimate the final model chosen by the LARS. I understand that the ...
4
votes
1answer
302 views

Classification with Least Angle (LARS)-type algorithm?

I am currently working on the LARS (Least Angle Regression) method. I know it is a regression method, but I wonder if, like LASSO or Ridge techniques (e.g. the package gmlnet in R), it can be ...
2
votes
0answers
75 views

question in least angle regression paper [Efron et al. (2004)]

I am reading Efron et al. (2004), Ann. Statist., 32, 2, "Least angle regression". I have struggled with formula 5.19 for some days. It says the minimizing resolution of lasso occurs strictly inside ...
0
votes
0answers
471 views

Inconsistent results for LASSO (glmnet package)

I am getting inconsistent results (coefficients and the mean cross-validated error) with the glmnet package in R. The data set has 33 variables and 250 observations. This is the sample output for the ...
4
votes
1answer
529 views

Why under joint least squares direction is it possible for some coefficients to decrease in LARS regression? [duplicate]

I think I understand how LARS regression works. It basically adds features to the model when they are more correlated with the residuals than the current model. ...
1
vote
1answer
767 views

Cross-validation and LASSO plot

I am not sure about some parts of the following plots: -- I do not understand what does the expression on x-axis means (first figure) and how to interpret that. -- What is the bars and the points in ...
3
votes
1answer
951 views

LARS vs LASSO and Cross-validation

I would like to apply lars algorithm to some datadset. First, I fitted the model to the training set and then examined it on test set. My questions: 1- After I used cross validation "cv.lars" I ...
1
vote
3answers
904 views

What is “step size” in LASSO regression?

I'm looking at this example for LASSO regression in R: http://machinelearningmastery.com/penalized-regression-in-r/. It ...
1
vote
1answer
158 views

Cross-validated prediction error worse for LARS than ordinary linear regression

I am analysing microarray data in order to build a model for predicting cell proliferation (a continuous variable) based on gene expression (also a continuous variable). There are many more genes than ...
3
votes
1answer
576 views

Computational complexity of the lasso (lars vs coordinate descent)

The lasso can be computed with the LARS or Coordinate Descent algorithm. What is their computational complexity and when one is quicker than the other?
8
votes
1answer
858 views

LASSO regularisation parameter from LARS algorithm

In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths. I have implemented this ...
1
vote
0answers
127 views

How to imagine (visualize) the difference between LARS and Lasso

I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact ...
1
vote
0answers
132 views

Convergence analysis for forward stagewise regression?

Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor $\hat{...
3
votes
0answers
432 views

R - CV.GLMNET LASSO (binomial) variable reduction - highly correlated variables not zeroing out

I have a dataset of about n = 100,000 observations and p = 247 predictors with one binomial dependent variable (values are 0, 1) I run the following code in R: ...
0
votes
1answer
717 views

lasso - how to evaluate results

I'm working on lasso as an alternative to step-wise forward/backward regression using the lars package in R. I normalized my variables, calculated the ...
0
votes
0answers
33 views

explaining LARS algorithm in a simple way [duplicate]

I am thinking to write a LARS algorithm that uses different optimizations in each step. Can somebody briefly explain Least angel regression, LARS (see here) to me? then I will try to write my own LARS ...
1
vote
1answer
6k views

How to interprete lasso from lars correctly?

I tried the lars package with R and got the following result. ...
1
vote
0answers
201 views

how to combine coefficients of a logistic regression model with existing prior knowledge about covariates?

I am working on developing statistical models for fault-localization. on the one hand, i construct a logistic regression model with these considerations: 1-my dependent(response) variable is program ...
11
votes
1answer
6k views

R - Lasso Regression - different Lambda per regressor

I want to do the following: 1) OLS regression (no penalization term) to get beta coefficients $b_{j}^{*}$; $j$ stands for the variables used to regress. I do this by ...
4
votes
1answer
2k views

Why is there no intercept in the lars output for LASSO in Stata?

This is my first time using lars, so this question is probably obvious. When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept. I ...
10
votes
2answers
5k views

Exact definition of Deviance measure in glmnet package, with crossvalidation?

For my current reseach I'm using the Lasso method via the glmnet package in R on a binomial dependent variable. In glmnet the optimal lambda is found via cross-validation and the resulting models ...
2
votes
2answers
1k views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
1
vote
0answers
129 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ \...
2
votes
1answer
142 views

Dataset for Least Angle Regression

I have read that least angle regression is good for high dimensional data. I didn't actually understand the meaning of high dimensional data, so does this mean $p>>n$ case? And does anyone know ...
5
votes
0answers
296 views

What is least angle regression?

Conceptually, I don't understand what least angle regression Least Angle Regression (LARS) is and why it solves LASSO (pdf). We know that LASSO is: $$\arg \min_x {\left\| A x - y \right\|}_{2}^{2} +...
1
vote
0answers
818 views

Parameter tuning in lars (lasso) matlab

I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm). I want to do a leave one out cross validation on my data using this code. I have the following ...
0
votes
0answers
259 views

How to add a non-negative constraint to lasso4j?

Lasso4j is a Java implementation of the Lasso L1-constrained fitting for linear regression. I would like to add a non-negativity constraint on the weights, meaning that the non-zero sparse ...
2
votes
3answers
309 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in \{1,...,1000\}\}...
39
votes
5answers
88k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
1answer
1k views

Least angle regression packages for R or MATLAB

I am looking for a Least Angle Regression (LAR) packages in R or MATLAB which can be used for classification problems. The only package that I currently know which fits this description is glmpath. ...
1
vote
1answer
508 views

'Forward Stagewise' option in LARS algorithm

Can anyone help me understand the forward stagewise part in the LARS algorithm? I was reading the R code and could not figure out what is ...
0
votes
1answer
189 views

Max steps in lars

I don't know how many steps are necessary for lars() to select the variables till the algorithm proceeds to the saturated fit (especially using the ...
2
votes
0answers
87 views

Constant signs of correlation in the active set in least angle regression

I am trying to comprehend the proof of the Least Angle Regression algorithm and I am stuck at certain points. I would appreciate any help that I can get. Let me set the stage: I am following the ...
7
votes
1answer
2k views

LARS - LASSO with weights

I am interested in solving the following problem $$ \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + \...
26
votes
2answers
2k views

Advantages of doing “double lasso” or performing lasso twice?

I once heard a method of using the lasso twice (like a double-lasso) where you perform lasso on the original set of variables, say S1, obtain a sparse set called S2, and then perform lasso again on ...
15
votes
2answers
720 views

LASSO/LARS vs general to specific (GETS) method

I have been wondering, why are LASSO and LARS model selection methods so popular even though they are basically just variations of step-wise forward selection (and thus suffer from path dependency)? ...
59
votes
4answers
13k views

What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
4
votes
1answer
616 views

Feature selection with k-fold cross-validated least angle regression

I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$). I have seven predictors ($x_1,x_2,...,x_7$) for each ...