A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

0
votes
0answers
29 views

How to derive this problem with soft-thresholding method?

The problem is defined as $$ \min_{x} \Bigg\{ a{\|x\|}^2+\frac{b}{2}{\|x-c\|}^2 \Bigg\} $$ where $x\in R^{n \times 1}, c \in R^{n \times 1}$ and $a,b$ are scalars. Equations 2.5 to 2.8 of this paper ...
1
vote
2answers
64 views

Why is Lasso regression for high dimensional data better than Stepwise AIC?

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So ...
0
votes
1answer
46 views

What is the practical significance of a linear regression, lasso regression, and ridge regression outputting the same coefficients?

The lasso and ridge regression are tuned to the same alpha parameter. No matter what I tune the parameter to [0,1], the results of all three regressions are always the same (linear, ridge, lasso), ...
2
votes
0answers
25 views

About the derivation of group Lasso

I've been reading the paper of group lasso, "Model selection and estimation in regression with grouped variables". http://www.stat.washington.edu/courses/stat527/s13/readings/yuanlin07.pdf In page 53 ...
1
vote
2answers
25 views

What is the meaning of regularization path in Lasso or related sparsity problems?

If we select different sparsity levels (i.e. that the $\lambda$ controls), we could obtain different solutions with different sparsity levels. Does it mean the regularization path is how to select the ...
5
votes
1answer
53 views

What's the typical range of possible values for the shrinkage parameter in penalized regression?

In lasso or ridge regression, one has to specify a shrinkage parameter, often called by $\lambda$ or $\alpha$. This value is often chosen via cross validation by checking a bunch of different values ...
1
vote
1answer
25 views

How to Find Adjusted $R^2$ or $R^2$ from Lasso and Ridge regression model

How do I find the adjusted $R^2$ (or $r^2$) from Lasso and Ridge regression? I used the glmnet package. For instance if I have this code so far.... ...
1
vote
1answer
38 views

How do you interpret the parameters obtained from lasso logistic regression when the response is binary?

Are we still able to interpret the parameters in the same manner as we would in ordinary logistic regression? I'm asking this because I'm toying with the german credit data ...
1
vote
1answer
33 views

LASSO or other regularized regression with censored (missing) data

Here is my problem. I am looking at various time series curves. Let's call them total spend aggregated over all customers on various products versus time. At any given time, I want to predict the ...
0
votes
1answer
63 views

How do you pronounce “LASSO”?

Some people pronounce it "LAS-so" and some "las-SOO" (from the rope cowboys throw to catch cows). Which way is "the right one"?
1
vote
0answers
31 views

lmmlasso does not work for p>n

I am using the following example from the R manual: ...
6
votes
1answer
66 views

How to decide which penalty measure to use ? any general guidelines or thumb rules out of textbook

A number of regularization measures are available in literatures, which is kind of confusing to beginners. The classical penalty is ridge by Hoerl & Kennard (1970,Technometrics 12, 55–67). ...
1
vote
0answers
64 views

How to deal with “not applicable” values in categorical variables

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
2
votes
1answer
95 views

large variables and low sample (p > n) problem: ridge , LASSO, PLS, PCR which is most suitable for predictions

I am trying see whether to go for ridge regression, LASSO or principal component regression (PCR) or Partial Least Squares (PLS) in a situation where there are large number of variables / features (p) ...
5
votes
2answers
141 views

AIC, BIC and GCV: what is best for making decision in penalized regression methods?

My general understanding is AIC deals with the trade-off between the goodness of fit of the model and the complexity of the model. $AIC =2k -2ln(L)$ $k$ = number of parameters in the model $L$ = ...
3
votes
1answer
84 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
1
vote
1answer
20 views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
11
votes
2answers
143 views

Bayesian lasso vs ordinary lasso

Different implementation software are available for lasso. I know a lot discussed about bayesian approach vs frequentist approach in different forums. My question is very specific to lasso - What are ...
2
votes
1answer
53 views

LASSO in the data with large number of variables (p) with lower number of samples (n)

I would like to fit LASSO in the following type of data where there are large number of variables (p > n). My y variable is y ...
1
vote
0answers
11 views

Repeated multiple regression for LASSO significance testing

I'm currently working on a statistical modelling problem in biology. We have cellular measurements of proteins in every cell in a tissue, and I'm using regression analysis to see if a given protein is ...
1
vote
0answers
19 views

GLS, heteroskedasticity and Ridge Regression/Lasso

I am hoping to use a regularised regression technique, using cross validation, to fit a linear model to a set of predictors which have some highly correlated variables. However, I also know (highly ...
2
votes
0answers
36 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
1
vote
0answers
32 views

Cross-Validation vs. AICc for LASSO

I was working on a research project in which I try to estimate the the individual contribution of a group of regional political leaders to local economic growth. The major challenge is that there is ...
0
votes
0answers
26 views

implementing Lasso with BIC

Do you have an R code to implement Lasso with BIC? Note that there is an R package called ...
4
votes
3answers
289 views

How to handle with missing values in order to prepare data for feature selection with LASSO?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
0
votes
0answers
51 views

Is it statistically sound to use Lasso for variable selection even when $n\gg p$?

I have a classification task where $n\gg p$ (like 440000 vs. 23). I want to use Lasso (glmnet in R) to select the variables first, then use techniques like random ...
3
votes
0answers
79 views

Generalized linear model with lasso regularization for continuous non-negative response

I have a big data problem with a large number of predictors and a non-negative response (time until inspection). For a full model I would use a glm with Gamma distributed response (link="log"). ...
0
votes
0answers
45 views

Penalized Bayesian quantile regression with LASSO and adaptive LASSO penalty

I have some questions about penalized Bayesian quantile regression with LASSO and adaptive LASSO penalty: Would you give me a detailed outline for the formula, especially as used in Bayesian ...
3
votes
2answers
94 views

What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
3
votes
0answers
39 views

What is a least angle regression?

Conceptually, I don't understand what is a least angle regression and why it solves LASSO http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf We know that LASSO is $$\min_x||Ax - ...
0
votes
0answers
104 views

“Convergence for 1st lambda value not reached”-error using GLMNET package and specifying lambda parameter

I get a weird problem when I specify lambda in the function glmnet, that does not appear if I let the function go through all the lambdas. When I run the following lines, it works great: ...
4
votes
1answer
181 views

How does glmnet handle overdispersion?

I have a question about how to model text over count data, in particular how could I use the lasso technique to reduce features. Say I have N online articles and ...
0
votes
1answer
61 views

Determine penalty parameter based on AIC values in lasso regression for linear mixed model

I have a dataset with 14 variables, of which one random effect (Patient) I fitted following code: "fit1_B2=lmmlasso(y=y,x=x.matrix,z=z,grp=grp,lambda=0.1,pdMat="pdIdent") summary(fit1_B2) ...
0
votes
1answer
30 views

use of validation set on lasso cross validation

When training a model a train, a validation and test set are used. I was wondering if there is any paper or example that proves that the use of an independent validation set increase the performance ...
0
votes
0answers
27 views

Laplace errors and ridge regression

Thinking about a normal linear regression, penalized by the L1 norm. That is the lasso, is there any literature on median ridge regression?. That is the residuals are calculated from an L1 norm ...
2
votes
1answer
42 views

How do I report Lasso results in an article?

I am using Lasso to reduce my number of variables and I have identified which variables had been retained after running my analysis (e.g. yrseduc, age). However, I don't know which numbers I should ...
2
votes
0answers
77 views

Lasso regression with GAM models in R

I'm using a regression package that uses gam() from the mgcv package. Is it possible to include an L1 penalty with the ...
1
vote
0answers
115 views

K-fold Cross Validation and Training/CV/Test set Techniques for choosing regularization parameter of Regression

Suppose I want to fit a lasso/ridge regression to a training set. Then, I need to choose $\lambda$, the regularization parameter. To choose $\lambda$, I can use two methods: K-fold Cross Validation ...
0
votes
0answers
12 views

sequence of regularization values in cross-validation

Does anybody know how cv.glmnet (R's package glmnet) or LassoCV (scikit-learn) chooses a sequence of regularization constants, which it uses in cross-validation? Are there any recommendations on how ...
1
vote
0answers
31 views

Regression model selection when there are more variables than cases

I have a database with 200+ variables and less then 50 cases. I need to choose an optimal model that predicts one dependent variable. Are stepwise/lasso regressions still appropriate methods to ...
1
vote
0answers
38 views

Least squares with multiple constraints

I work with a regression with ARMA errors and I want to use LASSO to shrink the coefficients and select my variables. This topic is discussed in the article Wu et al. (2012). So, the problem I have to ...
2
votes
1answer
61 views

Interpreting LASSO tables in SAS

I have been working on LASSO in SAS lately, and I'm still trying to figure out how to work with the options, but my main question for which I have not been able to find an answer on the internet so ...
0
votes
1answer
119 views

Help interpret cv.glmnet behavior for very large set of potential predictors

I've created this toy example to demonstrate something that is occurring with my real data. I do not understand how to interpret the apparent failure of cv.glmnet to find a solution when it's ...
2
votes
0answers
42 views

Importance of multivariate normality assumption for BIC-like sparse model selection inference with PCA

I am reading a paper for robust, sparse PCA in which they propose a BIC-like criterion for selecting the appropriate value of the sparsity parameter $\lambda$. They define this as: ...
1
vote
0answers
80 views

Parameter tuning in lars (lasso) matlab

I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm). I want to do a leave one out cross validation on my data using this code. I have the following ...
7
votes
2answers
238 views

Selection of k knots in regression smoothing spline equivalent to k categorical variables?

I'm working on a predictive cost model where the patient's age (an integer quantity measured in years) is one of the predictor variables. A strong nonlinear relationship between age and risk of a ...
1
vote
2answers
102 views

Strict convexity of Ridge vs Convexity of LASSO

Is there any intuition why the ridge regression is strictly convex, while the LASSO is only convex? Does it have to do with the "corners" of the L1 regularization?
2
votes
0answers
42 views

How can I tell if I my sample size is large enough for reliable feature selection using LASSO regression?

I have a gene expression dataset with 20 samples, and am not going to be getting any more. There are ~28,000 genes and four clinical covariates associated with each sample. The gene expression values ...
3
votes
1answer
282 views

Ridge, lasso and elastic net

How do ridge, LASSO and elasticnet regularization methods compare? What are their respective advantages and disadvantages? Any good technical paper, or lecture notes would be appreciated as well.
1
vote
1answer
101 views

Why no mention of penalized regression techniques in Applied Logistic Regression, 3rd edition, by Hosmer, Lemeshow, and Sturdivant?

Just ordered this textbook, and Wow, the complete omission of this subject from an otherwise excellent reference on logistic regression is a bit surprising. The 2nd edition was published in 2000 - ...