Questions tagged [lars]
LARS stands for Least Angle Regression. It is a penalized estimation and feature selection technique for multiple regression.
50
questions
0
votes
0
answers
27
views
Least angle regression algorithm in terms of gradient in lasso
I'm trying to understand an algorithm for a minimization problem but it is unclear. Here is the function we consider:
$\lVert Y - X\beta\rVert_{2}^{2} + \lambda\lVert\beta\rVert_{1}$ where $Y\in\...
- 111
1
vote
1
answer
170
views
When modified LARS implements LASSO, what's the corresponding lambda
We know that the modified LARS is used to implement LASSO. However what's the corresponding lambda? As there is no lambda parameter in LARS.
I found reference here:
LASSO regularisation parameter from ...
- 1,245
0
votes
0
answers
140
views
LARS package in R with specific lambda value
I am trying to use the LARS package in R to obtain a Lasso estimate of the sparse coefficient vector, say $\hat{\beta}_{\text{sparse}}$, as opposed to a coefficient path. In other words, I am not ...
- 1,144
0
votes
0
answers
170
views
glmnet vs. lars for computing the lasso
I've seen this post as well as this one regarding the difference between the lars and glmnet solution paths for fitting the ...
- 233
0
votes
0
answers
49
views
Model fit after inverse regression and forward stagewise on residuals
I work with prediction of height based on genetic information. The data is several hundred thousand data points from every organism, but a simplified version could be:
...
- 1
0
votes
1
answer
299
views
What does lars return for lambda equal to zero when p is larger than n?
For the lasso problem
$$
\hat \beta(\lambda) = \underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}}\|Y - X \beta\|_2^2 + \lambda \|\beta\|_1,
$$
where $Y$ is $p \times 1$, $X$ is $n \times p$, it ...
- 170
0
votes
1
answer
110
views
alpha and cv_alpha parameters in sklearn.linear_model.LarsCV
Can someone explain alpha and cv_alpha parameters in sklearn.linear_model.LarsCV? I am guessing that alphas refer to maximum correlation at any given step between one of the remaining explanatory ...
- 1
0
votes
2
answers
1k
views
using the Lasso in sklearn [closed]
I am a beginner, and I am trying to use the Lasso to do some regression. I am looking specifically at the LassoLars module in sklearn. What I am really after is recovering the parameter weight vector ...
- 377
1
vote
0
answers
30
views
Cross-validated methods as a default
Scikit-learn ( http://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model ) provide classes for Lasso, ElaticNet and Lars etc both with and without cross-validation.
What are ...
- 9,507
0
votes
0
answers
350
views
Drawbacks of Lasso-lars combined with recursive feature elimination
During the model selection phase of my (regression) work I noticed that Lars with lasso modification performs way worse than any other model if combined with recursive feature elimination, both in ...
- 147
1
vote
0
answers
247
views
Stacking Lasso models
let say I have M subsets of independent variables and I want to use stack learner to predict dependent variable y.
for each subset I use lasso method to get meta features (predictions).
I have 2 ...
- 111
2
votes
1
answer
669
views
How to get the equiangular vector in p dimension linear space? Used in Least angle regression
Here I am reading the "Least angle regression" by Efron, 2004. https://projecteuclid.org/download/pdfview_1/euclid.aos/1083178935
, in page 413, he gives formula of equiangular vector of the linear ...
- 23
2
votes
0
answers
113
views
How can lasso CCA be solved using LARS?
According to paper By Sun, Ji an Ye; A Least Squares Formulation for Canonical Correlation Analysis http://www.machinelearning.org/archive/icml2008/papers/270.pdf CCA can be reformulated as a least ...
- 7,341
5
votes
1
answer
619
views
What is the rationale behind LARS-OLS hybrid, i.e. using OLS estimate on the variables chosen by LARS?
I need some help to understand the relationship between the ranking of the variables from the LARS algorithm and the use of OLS to estimate the final model chosen by the LARS.
I understand that the ...
- 51
4
votes
1
answer
641
views
Classification with Least Angle (LARS)-type algorithm?
I am currently working on the LARS (Least Angle Regression) method.
I know it is a regression method, but I wonder if, like LASSO or Ridge techniques (e.g. the package gmlnet in R), it can be ...
- 41
2
votes
0
answers
149
views
question in least angle regression paper [Efron et al. (2004)]
I am reading Efron et al. (2004), Ann. Statist., 32, 2, "Least angle regression". I have struggled with formula 5.19 for some days. It says the minimizing resolution of lasso occurs strictly inside ...
- 21
0
votes
0
answers
677
views
Inconsistent results for LASSO (glmnet package)
I am getting inconsistent results (coefficients and the mean cross-validated error) with the glmnet package in R. The data set has 33 variables and 250 observations. This is the sample output for the ...
- 251
4
votes
1
answer
1k
views
Why under joint least squares direction is it possible for some coefficients to decrease in LARS regression? [duplicate]
I think I understand how LARS regression works. It basically adds features to the model when they are more correlated with the residuals than the current model. ...
- 2,239
1
vote
1
answer
2k
views
Cross-validation and LASSO plot
I am not sure about some parts of the following plots:
-- I do not understand what does the expression on x-axis means (first figure) and how to interpret that.
-- What is the bars and the points in ...
- 147
3
votes
1
answer
2k
views
LARS vs LASSO and Cross-validation
I would like to apply lars algorithm to some datadset.
First, I fitted the model to the training set and then examined it on test set.
My questions:
1- After I used cross validation "cv.lars" I ...
- 147
2
votes
3
answers
2k
views
What is "step size" in LASSO regression?
I'm looking at this example for LASSO regression in R: http://machinelearningmastery.com/penalized-regression-in-r/.
It ...
- 2,239
1
vote
1
answer
285
views
Cross-validated prediction error worse for LARS than ordinary linear regression
I am analysing microarray data in order to build a model for predicting cell proliferation (a continuous variable) based on gene expression (also a continuous variable). There are many more genes than ...
4
votes
1
answer
1k
views
Computational complexity of the lasso (lars vs coordinate descent)
The lasso can be computed with the LARS or Coordinate Descent algorithm.
What is their computational complexity and when one is quicker than the other?
- 3,129
10
votes
1
answer
2k
views
LASSO regularisation parameter from LARS algorithm
In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths.
I have implemented this ...
- 1,496
1
vote
0
answers
164
views
How to imagine (visualize) the difference between LARS and Lasso
I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact ...
- 91
2
votes
0
answers
294
views
Convergence analysis for forward stagewise regression?
Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor $\hat{...
- 238
3
votes
0
answers
597
views
R - CV.GLMNET LASSO (binomial) variable reduction - highly correlated variables not zeroing out
I have a dataset of about n = 100,000 observations and p = 247 predictors with one binomial dependent variable (values are 0, 1)
I run the following code in R:
...
Rushabh
0
votes
1
answer
2k
views
lasso - how to evaluate results
I'm working on lasso as an alternative to step-wise forward/backward regression using the lars package in R. I normalized my variables, calculated the ...
- 1
0
votes
0
answers
54
views
explaining LARS algorithm in a simple way [duplicate]
I am thinking to write a LARS algorithm that uses different optimizations in each step. Can somebody briefly explain Least angel regression, LARS (see here) to me? then I will try to write my own LARS ...
- 2,295
2
votes
1
answer
8k
views
How to interprete lasso from lars correctly?
I tried the lars package with R and got the following result.
...
- 705
1
vote
0
answers
372
views
how to combine coefficients of a logistic regression model with existing prior knowledge about covariates?
I am working on developing statistical models for fault-localization.
on the one hand, i construct a logistic regression model with these considerations:
1-my dependent(response) variable is program ...
- 11
12
votes
1
answer
7k
views
R - Lasso Regression - different Lambda per regressor
I want to do the following:
1) OLS regression (no penalization term) to get beta coefficients $b_{j}^{*}$; $j$ stands for the variables used to regress. I do this by
...
- 383
5
votes
1
answer
2k
views
Why is there no intercept in the lars output for LASSO in Stata?
This is my first time using lars, so this question is probably obvious.
When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept.
I ...
- 1,170
15
votes
3
answers
11k
views
Exact definition of Deviance measure in glmnet package, with crossvalidation?
For my current reseach I'm using the Lasso method via the glmnet package in R on a binomial dependent variable.
In glmnet the optimal lambda is found via cross-validation and the resulting models ...
- 173
3
votes
2
answers
2k
views
CV for LASSO tuning parameter using LARS
If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
- 650
1
vote
0
answers
172
views
Using Leave-One-Out Cross Validation with LARS
I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such:
$$\mathbb{min}\ \...
- 183
2
votes
1
answer
242
views
Dataset for Least Angle Regression
I have read that least angle regression is good for high dimensional data. I didn't actually understand the meaning of high dimensional data, so does this mean $p>>n$ case?
And does anyone know ...
- 123
5
votes
0
answers
474
views
What is least angle regression?
Conceptually, I don't understand what least angle regression Least Angle Regression (LARS) is and why it solves LASSO (pdf).
We know that LASSO is:
$$\arg \min_x {\left\| A x - y \right\|}_{2}^{2} +...
- 101
1
vote
0
answers
949
views
Parameter tuning in lars (lasso) matlab
I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm).
I want to do a leave one out cross validation on my data using this code. I have the following ...
- 175
3
votes
3
answers
360
views
What if Lasso selects transformed terms but not untransformed terms
Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in \{1,...,1000\}\}...
- 681
45
votes
5
answers
119k
views
Using LASSO from lars (or glmnet) package in R for variable selection
Sorry if this question comes across a little basic.
I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
- 451
1
vote
1
answer
1k
views
Least angle regression packages for R or MATLAB
I am looking for a Least Angle Regression (LAR) packages in R or MATLAB which can be used for classification problems.
The only package that I currently know which fits this description is glmpath. ...
- 4,745
1
vote
1
answer
710
views
'Forward Stagewise' option in LARS algorithm
Can anyone help me understand the forward stagewise part in the LARS algorithm? I was reading the R code and could not figure out what is ...
- 13
0
votes
1
answer
272
views
Max steps in lars
I don't know how many steps are necessary for lars() to select the variables till the algorithm proceeds to the saturated fit (especially using the ...
- 19
3
votes
0
answers
105
views
Constant signs of correlation in the active set in least angle regression
I am trying to comprehend the proof of the Least Angle Regression algorithm and I am stuck at certain points. I would appreciate any help that I can get.
Let me set the stage:
I am following the ...
- 391
8
votes
1
answer
3k
views
LARS - LASSO with weights
I am interested in solving the following problem
$$ \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + \...
- 391
30
votes
2
answers
5k
views
Advantages of doing "double lasso" or performing lasso twice?
I once heard a method of using the lasso twice (like a double-lasso) where you perform lasso on the original set of variables, say S1, obtain a sparse set called S2, and then perform lasso again on ...
- 871
15
votes
2
answers
978
views
LASSO/LARS vs general to specific (GETS) method
I have been wondering, why are LASSO and LARS model selection methods so popular even though they are basically just variations of step-wise forward selection (and thus suffer from path dependency)?
...
- 151
71
votes
5
answers
17k
views
What problem do shrinkage methods solve?
The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
- 13.8k
5
votes
1
answer
728
views
Feature selection with k-fold cross-validated least angle regression
I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$).
I have seven predictors ($x_1,x_2,...,x_7$) for each ...
- 1,969