# Tagged Questions

33 views

### How to report most important predictors using glmnet?

I want to find the most important predictors for a binomial dependent variable out of a set of more than 43,000 independent variables (These form the columns of my input dataset). The number of ...
111 views

### Are penalized regression techniques greedy algorithms?

In other words, is it feasible any of the various penalized regression techniques (such as ridge regression, lasso, and elasticnet) could completely miss the optimal solution for a regression model ...
46 views

### Confusion while using lassoglm

I am trying to fit a logistic regression model with L1 regularization on my data. My data has just 12 examples with 150 features. So I used L1 regularization. Now when I use the lassoglm function like ...
83 views

### Can the bias introduced by lasso change the sign of a coefficient?

L1 penalized regression introduces a bias on your regression model but decreases the variance. When this bias is introduced, is it possible that the coefficient of $B$ changes sign? This would ...
102 views

### Dantzig Selector, LASSO, LAD LASSO

I am wondering about this. When is it best to use Dantzig Selector (the infinity normed error measure plus the L1 regularizer) , the LASSO (the mean square error measure plus the L1 regularizer), and ...
87 views

### Logistic regression without negative samples

I have a data set of RNA reaction values of breast cancer. I want to figure out which RNAs are essential genes by Logistic Regression & LASSO. The data set has no negative samples. What should I ...
115 views

### Logistic regression model for analysis of many IVs with a relatively small sample size

I'm trying to determine the influence (direction and relative strength) of certain attributes of incoming students to an academic program on their successful completion of the program. My sample size ...
276 views

### If p > n, the lasso selects at most n variables

One of the motivations for the elastic net was the following limitation of LASSO: "In the p > n case, the lasso selects at most n variables before it saturates, because of the nature of the convex ...
121 views

### Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
829 views

166 views

### Model function for discovering irrelevant dimensions with L1 regularization

For homework I have been given a 20-dimensional input $x \in \mathbb{R}^{20}$, many of which are suspected to be irrelevant. I tried using L1-norm Lasso regularization to uncover which dimensions ...
685 views

### Why does Lasso do better than SVM?

This is a soft-question: I have been evaluation various regression techniques over a regression dataset that I have. I am surprised by the fact that cross-validated RMSE of Lasso is better than SVM ...
160 views

### Least angle regression for a set of vectors?

As far as I know, LARS solves the following problem (using the same notation as Efron et al. Least angle regression): Given a vector y, and a matrix X. Pick some column vectors from X, and express ...
325 views

### Bayesian prior corresponding to penalized regression coefficients

I'm working on a Bayesian Regression problem where I would like to estimate the beta coefficients subject to this constraint (penalty): $\sum|\beta_i|<C$ or similarly $\sum \beta_i^2<C$ Which ...
600 views

### Need for centering and standardizing data in regression

Consider linear regression with some regularization: E.g. Find $x$ that minimizes $||Ax - b||^2+\lambda||x||_1$ Usually, columns of A are standardized to have zero mean and unit norm, while $b$ is ...
3k views

### What is the lasso in regression analysis?

I'm looking for a non-technical definition of the lasso and what it is used for.
680 views

### Cox model with LASSO

Rob Tibshirani propose to use lasso with Cox regression for variable selection in his 1997 paper "The lasso method for variable selection in the Cox model" published in Statistics In Medicine 16:385. ...
149 views

### Regularized fit from summarized data

I have a multiple linear regression problem $y=X\beta+\epsilon$. The number of observations $m$ is large, so by the time the data gets to me it's been summarized into: $m$ $X^TX$ $X^Ty$ $y^Ty$ ...
2k views

### What are disadvantages of using the lasso for variable selection for regression?

From what I know, using lasso for variable selection handles the problem of correlated inputs. Also, since it is equivalent to Least Angle Regression, it is not slow computationally. However, many ...
2k views

### Estimating R-squared and statistical significance from penalized regression model

I am using the R package penalized to obtain shrunken estimates of coefficients for a dataset where I have lots of predictors and little knowledge of which ones are important. After I've picked tuning ...
3k views

### GLMNET or LARS for computing LASSO solutions?

I would like to get the coefficients for the LASSO problem $$||Y-X\beta||+\lambda ||\beta||_1.$$ The problem is that glmnet and lars functions give different answers. For the glmnet function I ask ...
577 views

### LASSO assumptions

In a LASSO regression scenario where $y= X \beta + \epsilon$, and the LASSO estimates are given by the following optimization problem $\min_\beta ||y - X \beta|| + \tau||\beta||_1$ Are there any ...
321 views

### Stochastic coordinate descent for $\ell_1$ regularization

I recently came across the following paper: "Stochastic Methods for $\ell_1$ Regularized Loss Minimization" by Shai Shalev-Shwartz and Ambuj Tewari, ICML 2009. In the paper, the authors propose a ...
903 views

### Lasso fitting by coordinate descent: open-source implementations?

What open-source implementations -- in any language -- exist out there that can compute lasso regularisation paths for linear regression by coordinate descent? So far I am aware of: glmnet ...
671 views

### LARS vs coordinate descent for the lasso

What are the pros and cons of using LARS [1] versus using coordinate descent for fitting L1-regularized linear regression? I am mainly interested in performance aspects (my problems tend to have ...
589 views

### Coordinate descent for the lasso or elastic net

Are there any good papers or books dealing with the use of coordinate descent for L1 (lasso) and/or elastic net regularization for linear regression problems?
2k views

### Least-angle regression vs. lasso

Least-angle regression and the lasso tend to produce very similar regularization paths (identical except when a coefficient crosses zero.) They both can be efficiently fit by virtually identical ...
193 views

### Updating the lasso fit with new observations

I am fitting an L1-regularized linear regression to a very large dataset (with n>>p.) The variables are known in advance, but the observations arrive in small chunks. I would like to maintain the ...
849 views

### Java implementations of the lasso

Are there any open-source Java implementations of lasso or least angles regression? Pure Java code would be best, but clean implementations in other languages would also be of interest. I am already ...
1k views

### In R, does “glmnet” fit an intercept?

I am fitting a linear model in R using glmnet. The original (non-regularized) model was fitted using lm and did not have a ...