Skip to main content
9 votes

What is the rationale behind LARS-OLS hybrid, i.e. using OLS estimate on the variables chosen by LARS?

The coefficient estimates from LARS will be shrunk (biased) towards zero, and the intensity of shrinkage might be suboptimal (too harsh) for forecasting. However, some shrinkage should be good, as ...
Richard Hardy's user avatar
6 votes

Why under joint least squares direction is it possible for some coefficients to decrease in LARS regression?

As the answer in the link (stats.stackexchange.com/questions/154870) provided by Ben says, it has mostly to do with the correlation between the columns of the $X$ matrix in the regression $y = X\beta +...
Sextus Empiricus's user avatar
4 votes

Computational complexity of the lasso (lars vs coordinate descent)

I realize it's quite late to give an answer but maybe someone will find it useful. Here is a nice talk by Trevor Hastie about the coordinate descent. He compares (among others) his two R packages: <...
CherryGarcia's user avatar
4 votes

using the Lasso in sklearn

_coef give you parameter vector (w in the cost function formula). You should look at y vector. ...
RobJan's user avatar
  • 198
3 votes
Accepted

How to get the equiangular vector in p dimension linear space? Used in Least angle regression

For ease of notation I'm simply going to consider the following problem: Given a matrix $X$ whose columns $x^{(j)}$ are linearly independent, each with mean zero and unit variance, we want to find a ...
Epiousios's user avatar
  • 238
3 votes

using the Lasso in sklearn

Issue 1 In addition to RobJan's answer, I think there is something unintended in your code: y = [np.mean(X)] * n This takes the mean of the whole matrix, and ...
danuker's user avatar
  • 179
2 votes

Exact definition of Deviance measure in glmnet package, with crossvalidation?

In addition to the @shadowtalker 's answer, when I was using the package glmnet, I feel like the deviance in the cross-validation is somehow normalized. ...
vtshen's user avatar
  • 449
2 votes

Classification with Least Angle (LARS)-type algorithm?

The only thing you would need to change is the inclusion of a link function to map the outcomes to the appropriate scale, i.e. $\mathbb{R}^k \to (0,1)^k$ and change the loss function to be a ...
Sycorax's user avatar
  • 94.1k
1 vote
Accepted

When modified LARS implements LASSO, what's the corresponding lambda

(Modified) LARS gives a sequence of coefficient estimates, call it $(\hat\beta_1,\hat\beta_2,\hat\beta_3,\dots\hat\beta_k)$. Lasso gives a path $(\tilde\beta_\lambda,\lambda)$ containing the optimal $\...
Thomas Lumley's user avatar
1 vote
Accepted

What does lars return for lambda equal to zero when p is larger than n?

In short, the answer is that it's the minimum $\ell_1$ norm least squares estimator. This was shown in Lemma 7 of: Tibshirani, R. J. (2013). The lasso problem and uniqueness. Electronic Journal of ...
user257566's user avatar
1 vote

Exact definition of Deviance measure in glmnet package, with crossvalidation?

As @vtshen mentions, there must be a standardization of Deviance values in cv.glment. After tracing the function code provided by @shadowtalker, I have arrived at the line that corroborates the ...
Tomás Navarro's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible