9
votes
What is the rationale behind LARS-OLS hybrid, i.e. using OLS estimate on the variables chosen by LARS?
The coefficient estimates from LARS will be shrunk (biased) towards zero, and the intensity of shrinkage might be suboptimal (too harsh) for forecasting.
However, some shrinkage should be good, as ...
6
votes
Why under joint least squares direction is it possible for some coefficients to decrease in LARS regression?
As the answer in the link (stats.stackexchange.com/questions/154870) provided by Ben says, it has mostly to do with the correlation between the columns of the $X$ matrix in the regression $y = X\beta +...
4
votes
Computational complexity of the lasso (lars vs coordinate descent)
I realize it's quite late to give an answer but maybe someone will find it useful.
Here is a nice talk by Trevor Hastie about the coordinate descent. He compares (among others) his two R packages: <...
4
votes
using the Lasso in sklearn
_coef give you parameter vector (w in the cost function formula). You should look at y vector.
...
3
votes
Accepted
How to get the equiangular vector in p dimension linear space? Used in Least angle regression
For ease of notation I'm simply going to consider the following problem: Given a matrix $X$ whose columns $x^{(j)}$ are linearly independent, each with mean zero and unit variance, we want to find a ...
3
votes
using the Lasso in sklearn
Issue 1
In addition to RobJan's answer, I think there is something unintended in your code:
y = [np.mean(X)] * n
This takes the mean of the whole matrix, and ...
2
votes
Exact definition of Deviance measure in glmnet package, with crossvalidation?
In addition to the @shadowtalker 's answer, when I was using the package glmnet, I feel like the deviance in the cross-validation is somehow normalized.
...
2
votes
Classification with Least Angle (LARS)-type algorithm?
The only thing you would need to change is the inclusion of a link function to map the outcomes to the appropriate scale, i.e. $\mathbb{R}^k \to (0,1)^k$ and change the loss function to be a ...
1
vote
Accepted
When modified LARS implements LASSO, what's the corresponding lambda
(Modified) LARS gives a sequence of coefficient estimates, call it $(\hat\beta_1,\hat\beta_2,\hat\beta_3,\dots\hat\beta_k)$. Lasso gives a path $(\tilde\beta_\lambda,\lambda)$ containing the optimal $\...
1
vote
Accepted
What does lars return for lambda equal to zero when p is larger than n?
In short, the answer is that it's the minimum $\ell_1$ norm least squares estimator. This was shown in Lemma 7 of:
Tibshirani, R. J. (2013). The lasso problem and uniqueness. Electronic Journal of ...
1
vote
Exact definition of Deviance measure in glmnet package, with crossvalidation?
As @vtshen mentions, there must be a standardization of Deviance values in cv.glment. After tracing the function code provided by @shadowtalker, I have arrived at the line that corroborates the ...
Only top scored, non community-wiki answers of a minimum length are eligible
Related Tags
lars × 50lasso × 35
regression × 17
r × 10
cross-validation × 8
feature-selection × 6
glmnet × 6
regularization × 5
machine-learning × 4
model-selection × 3
scikit-learn × 3
logistic × 2
classification × 2
correlation × 2
matlab × 2
ridge-regression × 2
stepwise-regression × 2
sparse × 2
time-series × 1
self-study × 1
mathematical-statistics × 1
multiple-regression × 1
python × 1
estimation × 1
optimization × 1