A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

2
votes
1answer
36 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
2
votes
1answer
41 views

How is lasso an M-Estimator?

The definition of an M-estimator is an estimator (from Casella and Berger) of the form $$\hat{\theta}=\min \sum_{i=1}^n \rho(X_i-\theta),$$ where $X_1,X_2, \cdots, X_n$ is the data for some function ...
-2
votes
0answers
20 views

I need the whole Lasso(Least Absolute Shrinkage and Selection Operator) Algorithm. Anyone?

Anyone could give me the whole Lasso Algorithm? The step by step procedure in determining Lasso solutions? I barely need this for my thesis. Thank You.
2
votes
1answer
18 views

Variable selection: Why certain categories are chosen but not others?

I'm doing variable selection using the Lasso. To explain my response variable I have several predictors, both categorical and numerical, but I have problems to explain the process that underlies ...
0
votes
0answers
19 views

Code for $\ell_1$/ $\ell_2$ sparse multivariate regression algorithm

Where can I find the code (Matlab or R) for the block-regularized Lasso problem defined as follows: $\min_{B} \| Y-XB\|_F^2 + \lambda \|B\|_{1,2}$ where $Y$, $X$, and $B$ are matrices. This is a ...
0
votes
0answers
14 views

Apply LASSO Model with nominal target in SAS?

I'm building a classification model with a pool of independent variables (hundreds of them). I'm in the step of variable selection/feature selection. Now I'm trying to figure out if there are any ...
0
votes
1answer
35 views

lasso with missing values and categorical variables

I have a dataset with a lot of missing values and mix of continues and categorical variables. I want to use something like group lasso to do features selection. Probably the output is binary 0,1 and ...
0
votes
0answers
31 views

How to derive this problem with soft-thresholding method?

The problem is defined as $$ \min_{x} \Bigg\{ a{\|x\|}^2+\frac{b}{2}{\|x-c\|}^2 \Bigg\} $$ where $x\in R^{n \times 1}, c \in R^{n \times 1}$ and $a,b$ are scalars. Equations 2.5 to 2.8 of this paper ...
1
vote
2answers
77 views

Why is Lasso regression for high dimensional data better than Stepwise AIC?

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So ...
0
votes
1answer
54 views

What is the practical significance of a linear regression, lasso regression, and ridge regression outputting the same coefficients? [closed]

The lasso and ridge regression are tuned to the same alpha parameter. No matter what I tune the parameter to [0,1], the results of all three regressions are always the same (linear, ridge, lasso), ...
2
votes
0answers
29 views

About the derivation of group Lasso

I've been reading the paper of group lasso, "Model selection and estimation in regression with grouped variables". http://www.stat.washington.edu/courses/stat527/s13/readings/yuanlin07.pdf In page 53 ...
1
vote
2answers
26 views

What is the meaning of regularization path in Lasso or related sparsity problems?

If we select different sparsity levels (i.e. that the $\lambda$ controls), we could obtain different solutions with different sparsity levels. Does it mean the regularization path is how to select the ...
6
votes
1answer
60 views

What's the typical range of possible values for the shrinkage parameter in penalized regression?

In lasso or ridge regression, one has to specify a shrinkage parameter, often called by $\lambda$ or $\alpha$. This value is often chosen via cross validation by checking a bunch of different values ...
1
vote
1answer
36 views

How to Find Adjusted $R^2$ or $R^2$ from Lasso and Ridge regression model

How do I find the adjusted $R^2$ (or $r^2$) from Lasso and Ridge regression? I used the glmnet package. For instance if I have this code so far.... ...
1
vote
1answer
48 views

How do you interpret the parameters obtained from lasso logistic regression when the response is binary?

Are we still able to interpret the parameters in the same manner as we would in ordinary logistic regression? I'm asking this because I'm toying with the german credit data ...
1
vote
1answer
44 views

LASSO or other regularized regression with censored (missing) data

Here is my problem. I am looking at various time series curves. Let's call them total spend aggregated over all customers on various products versus time. At any given time, I want to predict the ...
0
votes
1answer
66 views

How do you pronounce “LASSO”?

Some people pronounce it "LAS-so" and some "las-SOO" (from the rope cowboys throw to catch cows). Which way is "the right one"?
1
vote
0answers
34 views

lmmlasso does not work for p>n

I am using the following example from the R manual: ...
6
votes
1answer
71 views

How to decide which penalty measure to use ? any general guidelines or thumb rules out of textbook

A number of regularization measures are available in literatures, which is kind of confusing to beginners. The classical penalty is ridge by Hoerl & Kennard (1970,Technometrics 12, 55–67). ...
1
vote
0answers
71 views

How to deal with “not applicable” values in categorical variables

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
2
votes
1answer
120 views

large variables and low sample (p > n) problem: ridge , LASSO, PLS, PCR which is most suitable for predictions

I am trying see whether to go for ridge regression, LASSO or principal component regression (PCR) or Partial Least Squares (PLS) in a situation where there are large number of variables / features (p) ...
5
votes
2answers
156 views

AIC, BIC and GCV: what is best for making decision in penalized regression methods?

My general understanding is AIC deals with the trade-off between the goodness of fit of the model and the complexity of the model. $AIC =2k -2ln(L)$ $k$ = number of parameters in the model $L$ = ...
3
votes
1answer
93 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
1
vote
2answers
32 views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
11
votes
2answers
153 views

Bayesian lasso vs ordinary lasso

Different implementation software are available for lasso. I know a lot discussed about bayesian approach vs frequentist approach in different forums. My question is very specific to lasso - What are ...
2
votes
1answer
59 views

LASSO in the data with large number of variables (p) with lower number of samples (n)

I would like to fit LASSO in the following type of data where there are large number of variables (p > n). My y variable is y ...
1
vote
0answers
11 views

Repeated multiple regression for LASSO significance testing

I'm currently working on a statistical modelling problem in biology. We have cellular measurements of proteins in every cell in a tissue, and I'm using regression analysis to see if a given protein is ...
1
vote
0answers
22 views

GLS, heteroskedasticity and Ridge Regression/Lasso

I am hoping to use a regularised regression technique, using cross validation, to fit a linear model to a set of predictors which have some highly correlated variables. However, I also know (highly ...
2
votes
0answers
37 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
1
vote
0answers
36 views

Cross-Validation vs. AICc for LASSO

I was working on a research project in which I try to estimate the the individual contribution of a group of regional political leaders to local economic growth. The major challenge is that there is ...
0
votes
0answers
28 views

implementing Lasso with BIC

Do you have an R code to implement Lasso with BIC? Note that there is an R package called ...
4
votes
3answers
318 views

How to handle with missing values in order to prepare data for feature selection with LASSO?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
0
votes
0answers
51 views

Is it statistically sound to use Lasso for variable selection even when $n\gg p$?

I have a classification task where $n\gg p$ (like 440000 vs. 23). I want to use Lasso (glmnet in R) to select the variables first, then use techniques like random ...
3
votes
0answers
88 views

Generalized linear model with lasso regularization for continuous non-negative response

I have a big data problem with a large number of predictors and a non-negative response (time until inspection). For a full model I would use a glm with Gamma distributed response (link="log"). ...
0
votes
0answers
52 views

Penalized Bayesian quantile regression with LASSO and adaptive LASSO penalty

I have some questions about penalized Bayesian quantile regression with LASSO and adaptive LASSO penalty: Would you give me a detailed outline for the formula, especially as used in Bayesian ...
3
votes
2answers
100 views

What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
3
votes
0answers
41 views

What is a least angle regression?

Conceptually, I don't understand what is a least angle regression and why it solves LASSO http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf We know that LASSO is $$\min_x||Ax - ...
0
votes
0answers
129 views

“Convergence for 1st lambda value not reached”-error using GLMNET package and specifying lambda parameter

I get a weird problem when I specify lambda in the function glmnet, that does not appear if I let the function go through all the lambdas. When I run the following lines, it works great: ...
4
votes
1answer
206 views

How does glmnet handle overdispersion?

I have a question about how to model text over count data, in particular how could I use the lasso technique to reduce features. Say I have N online articles and ...
0
votes
1answer
65 views

Determine penalty parameter based on AIC values in lasso regression for linear mixed model

I have a dataset with 14 variables, of which one random effect (Patient) I fitted following code: "fit1_B2=lmmlasso(y=y,x=x.matrix,z=z,grp=grp,lambda=0.1,pdMat="pdIdent") summary(fit1_B2) ...
0
votes
1answer
30 views

use of validation set on lasso cross validation

When training a model a train, a validation and test set are used. I was wondering if there is any paper or example that proves that the use of an independent validation set increase the performance ...
0
votes
0answers
28 views

Laplace errors and ridge regression

Thinking about a normal linear regression, penalized by the L1 norm. That is the lasso, is there any literature on median ridge regression?. That is the residuals are calculated from an L1 norm ...
2
votes
1answer
45 views

How do I report Lasso results in an article?

I am using Lasso to reduce my number of variables and I have identified which variables had been retained after running my analysis (e.g. yrseduc, age). However, I don't know which numbers I should ...
2
votes
0answers
84 views

Lasso regression with GAM models in R

I'm using a regression package that uses gam() from the mgcv package. Is it possible to include an L1 penalty with the ...
1
vote
0answers
131 views

K-fold Cross Validation and Training/CV/Test set Techniques for choosing regularization parameter of Regression

Suppose I want to fit a lasso/ridge regression to a training set. Then, I need to choose $\lambda$, the regularization parameter. To choose $\lambda$, I can use two methods: K-fold Cross Validation ...
0
votes
0answers
12 views

sequence of regularization values in cross-validation

Does anybody know how cv.glmnet (R's package glmnet) or LassoCV (scikit-learn) chooses a sequence of regularization constants, which it uses in cross-validation? Are there any recommendations on how ...
1
vote
0answers
31 views

Regression model selection when there are more variables than cases

I have a database with 200+ variables and less then 50 cases. I need to choose an optimal model that predicts one dependent variable. Are stepwise/lasso regressions still appropriate methods to ...
1
vote
0answers
42 views

Least squares with multiple constraints

I work with a regression with ARMA errors and I want to use LASSO to shrink the coefficients and select my variables. This topic is discussed in the article Wu et al. (2012). So, the problem I have to ...
2
votes
1answer
74 views

Interpreting LASSO tables in SAS

I have been working on LASSO in SAS lately, and I'm still trying to figure out how to work with the options, but my main question for which I have not been able to find an answer on the internet so ...
0
votes
1answer
121 views

Help interpret cv.glmnet behavior for very large set of potential predictors

I've created this toy example to demonstrate something that is occurring with my real data. I do not understand how to interpret the apparent failure of cv.glmnet to find a solution when it's ...