# Tagged Questions

Is a form of regularization used in estimation of regression coefficients which shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). The LASSO is equivalent to the Bayesian estimation problem where iid standard Laplacian prior is used for the ...

141 views

### Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
5k views

### How to present results of a Lasso using glmnet?

I would like to find predictors for a continuous dependent variable out of a set of 30 independent variables. I am using Lasso regression as implemented in the glmnet package in R. Here is some dummy ...
631 views

### L1 regression estimates median whereas L2 regression estimates mean?

So I was asked a question on which central measures L1 (i.e., lasso) and L2 (i.e., ridge regression) estimated. The answer is L1=median and L2=mean. Is there any type of intuitive reasoning to this? ...
484 views

### Variable selection with LASSO

I am trying to fit a predictive gene-based model in survival analysis. My question is: Can I use LASSO as a variable selection method, and then run a multivariate Cox regression to get the ...
1k views

168 views

### Model function for discovering irrelevant dimensions with L1 regularization

For homework I have been given a 20-dimensional input $x \in \mathbb{R}^{20}$, many of which are suspected to be irrelevant. I tried using L1-norm Lasso regularization to uncover which dimensions ...
856 views

### Why does Lasso do better than SVM?

This is a soft-question: I have been evaluation various regression techniques over a regression dataset that I have. I am surprised by the fact that cross-validated RMSE of Lasso is better than SVM ...
436 views

### How to find parameters for ridge and lasso regularization when cost minimization does not converge?

In the Stanford ML course, we were taught to find good values for the lambda parameters of ridge/lasso by iterating for various lambda values on several cross-validation sets and picking the values ...
7k views

### How to estimate shrinkage parameter in Lasso or ridge regression?

I want to use Lasso or ridge regression for a model with more than 50,000 variables. I want do so using software package in R. How can I estimate the shrinkage parameter ($\lambda$)? Edits: Here is ...
914 views

### Is it possible to calculate AIC and BIC for lasso regression models?

Is is possible to calculate an AIC or BIC values for lasso regression models and other regularized models where parameters are only partially entering the equation. How does one determine the degrees ...
527 views

### Optimal parameter selection by repeated k-fold

I am working on Lasso problem and the selection of the optimal tuning parameter with $k$-fold procedure, say $k=10$. Since this procedure relies on random subsampling, value of the optimal parameter ...
1k views

### Using glmnet to solve the LASSO problem

I have recently been made aware of the Lasso algorithm and found that the package glmnet can be used to solve it. (I have the glmnet package on R). If I have a matrix $A$ and a vector $y$ how do I ...
165 views

### Least angle regression for a set of vectors?

As far as I know, LARS solves the following problem (using the same notation as Efron et al. Least angle regression): Given a vector y, and a matrix X. Pick some column vectors from X, and express ...
413 views

### Bayesian prior corresponding to penalized regression coefficients

I'm working on a Bayesian Regression problem where I would like to estimate the beta coefficients subject to this constraint (penalty): $\sum|\beta_i|<C$ or similarly $\sum \beta_i^2<C$ Which ...
281 views

### LASSO/LARS vs general to specific (GETS) method

I have been wondering, why are LASSO and LARS model selection methods so popular even though they are basically just variations of step-wise forward selection (and thus suffer from path dependency)? ...
113 views

### Cox model with Lasso for PH test and P value

I am using the penalized package in r for fitting a Cox model with a lasso penalty. Does the Cox model + lasso need to test proportionality? How do you get the ...
715 views

### Can compressed sensing be applied to data mining?

For high dimensional data, LASSO is useful as it allows to determine the few significant covariates. I think compressed sensing is an upgrade of LASSO. According to the wiki page: [the LASSO ...
1k views

### What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
829 views

### Need for centering and standardizing data in regression

Consider linear regression with some regularization: E.g. Find $x$ that minimizes $||Ax - b||^2+\lambda||x||_1$ Usually, columns of A are standardized to have zero mean and unit norm, while $b$ is ...
327 views

### Lasso modification for LARS

I am trying to understand how Lars algorithm can be modified to generate Lasso. While I do understand LARS, I am not able to see the Lasso modification from the paper by Tibshirani et al. In ...
2k views

694 views

### LASSO assumptions

In a LASSO regression scenario where $y= X \beta + \epsilon$, and the LASSO estimates are given by the following optimization problem $\min_\beta ||y - X \beta|| + \tau||\beta||_1$ Are there any ...
354 views

### Stochastic coordinate descent for $\ell_1$ regularization

I recently came across the following paper: "Stochastic Methods for $\ell_1$ Regularized Loss Minimization" by Shai Shalev-Shwartz and Ambuj Tewari, ICML 2009. In the paper, the authors propose a ...
1k views

### Lasso fitting by coordinate descent: open-source implementations?

What open-source implementations -- in any language -- exist out there that can compute lasso regularisation paths for linear regression by coordinate descent? So far I am aware of: glmnet ...
789 views

### LARS vs coordinate descent for the lasso

What are the pros and cons of using LARS [1] versus using coordinate descent for fitting L1-regularized linear regression? I am mainly interested in performance aspects (my problems tend to have ...