A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

6
votes
1answer
59 views

How to decide which penalty measure to use ? any general guidelines or thumb rules out of textbook

A number of regularization measures are available in literatures, which is kind of confusing to beginners. The classical penalty is ridge by Hoerl & Kennard (1970,Technometrics 12, 55–67). ...
1
vote
0answers
32 views

How to deal with “not applicable” values in categorical variables

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
2
votes
1answer
39 views

large variables and low sample (p > n) problem: ridge , LASSO, PLS, PCR which is most suitable for predictions

I am trying see whether to go for ridge regression, LASSO or principal component regression (PCR) or Partial Least Squares (PLS) in a situation where there are large number of variables / features (p) ...
5
votes
2answers
103 views

AIC, BIC and GCV: what is best for making decision in penalized regression methods?

My general understanding is AIC deals with the trade-off between the goodness of fit of the model and the complexity of the model. $AIC =2k -2ln(L)$ $k$ = number of parameters in the model $L$ = ...
3
votes
1answer
63 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
1
vote
0answers
11 views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
8
votes
1answer
111 views
+50

Bayesian lasso vs ordinary lasso

Different implementation software are available for lasso. I know a lot discussed about bayesian approach vs frequentist approach in different forums. My question is very specific to lasso - What are ...
2
votes
1answer
44 views

LASSO in the data with large number of variables (p) with lower number of samples (n)

I would like to fit LASSO in the following type of data where there are large number of variables (p > n). My y variable is y ...
1
vote
0answers
5 views

Repeated multiple regression for LASSO significance testing

I'm currently working on a statistical modelling problem in biology. We have cellular measurements of proteins in every cell in a tissue, and I'm using regression analysis to see if a given protein is ...
1
vote
0answers
14 views

GLS, heteroskedasticity and Ridge Regression/Lasso

I am hoping to use a regularised regression technique, using cross validation, to fit a linear model to a set of predictors which have some highly correlated variables. However, I also know (highly ...
2
votes
0answers
33 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
1
vote
0answers
23 views

Cross-Validation vs. AICc for LASSO

I was working on a research project in which I try to estimate the the individual contribution of a group of regional political leaders to local economic growth. The major challenge is that there is ...
0
votes
0answers
17 views

implementing Lasso with BIC

Do you have an R code to implement Lasso with BIC? Note that there is an R package called ...
4
votes
3answers
231 views

How to handle with missing values in order to prepare data for feature selection with LASSO?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
0
votes
0answers
46 views

Is it statistically sound to use Lasso for variable selection even when $n\gg p$?

I have a classification task where $n\gg p$ (like 440000 vs. 23). I want to use Lasso (glmnet in R) to select the variables first, then use techniques like random ...
3
votes
0answers
63 views

Generalized linear model with lasso regularization for continuous non-negative response

I have a big data problem with a large number of predictors and a non-negative response (time until inspection). For a full model I would use a glm with Gamma distributed response (link="log"). ...
0
votes
0answers
35 views

Penalized Bayesian quantile regression with LASSO and adaptive LASSO penalty

I have some questions about penalized Bayesian quantile regression with LASSO and adaptive LASSO penalty: Would you give me a detailed outline for the formula, especially as used in Bayesian ...
3
votes
2answers
84 views

What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
3
votes
0answers
38 views

What is a least angle regression?

Conceptually, I don't understand what is a least angle regression and why it solves LASSO http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf We know that LASSO is $$\min_x||Ax - ...
0
votes
0answers
65 views

“Convergence for 1st lambda value not reached”-error using GLMNET package and specifying lambda parameter

I get a weird problem when I specify lambda in the function glmnet, that does not appear if I let the function go through all the lambdas. When I run the following lines, it works great: ...
3
votes
1answer
156 views

How does glmnet handle overdispersion?

I have a question about how to model text over count data, in particular how could I use the lasso technique to reduce features. Say I have N online articles and ...
0
votes
1answer
56 views

Determine penalty parameter based on AIC values in lasso regression for linear mixed model

I have a dataset with 14 variables, of which one random effect (Patient) I fitted following code: "fit1_B2=lmmlasso(y=y,x=x.matrix,z=z,grp=grp,lambda=0.1,pdMat="pdIdent") summary(fit1_B2) ...
0
votes
1answer
26 views

use of validation set on lasso cross validation

When training a model a train, a validation and test set are used. I was wondering if there is any paper or example that proves that the use of an independent validation set increase the performance ...
0
votes
0answers
25 views

Laplace errors and ridge regression

Thinking about a normal linear regression, penalized by the L1 norm. That is the lasso, is there any literature on median ridge regression?. That is the residuals are calculated from an L1 norm ...
2
votes
1answer
39 views

How do I report Lasso results in an article?

I am using Lasso to reduce my number of variables and I have identified which variables had been retained after running my analysis (e.g. yrseduc, age). However, I don't know which numbers I should ...
2
votes
0answers
73 views

Lasso regression with GAM models in R

I'm using a regression package that uses gam() from the mgcv package. Is it possible to include an L1 penalty with the ...
1
vote
0answers
84 views

K-fold Cross Validation and Training/CV/Test set Techniques for choosing regularization parameter of Regression

Suppose I want to fit a lasso/ridge regression to a training set. Then, I need to choose $\lambda$, the regularization parameter. To choose $\lambda$, I can use two methods: K-fold Cross Validation ...
0
votes
0answers
10 views

sequence of regularization values in cross-validation

Does anybody know how cv.glmnet (R's package glmnet) or LassoCV (scikit-learn) chooses a sequence of regularization constants, which it uses in cross-validation? Are there any recommendations on how ...
1
vote
0answers
30 views

Regression model selection when there are more variables than cases

I have a database with 200+ variables and less then 50 cases. I need to choose an optimal model that predicts one dependent variable. Are stepwise/lasso regressions still appropriate methods to ...
1
vote
0answers
31 views

Least squares with multiple constraints

I work with a regression with ARMA errors and I want to use LASSO to shrink the coefficients and select my variables. This topic is discussed in the article Wu et al. (2012). So, the problem I have to ...
2
votes
1answer
46 views

Interpreting LASSO tables in SAS

I have been working on LASSO in SAS lately, and I'm still trying to figure out how to work with the options, but my main question for which I have not been able to find an answer on the internet so ...
0
votes
1answer
111 views

Help interpret cv.glmnet behavior for very large set of potential predictors

I've created this toy example to demonstrate something that is occurring with my real data. I do not understand how to interpret the apparent failure of cv.glmnet to find a solution when it's ...
2
votes
0answers
38 views

Importance of multivariate normality assumption for BIC-like sparse model selection inference with PCA

I am reading a paper for robust, sparse PCA in which they propose a BIC-like criterion for selecting the appropriate value of the sparsity parameter $\lambda$. They define this as: ...
1
vote
0answers
64 views

Parameter tuning in lars (lasso) matlab

I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm). I want to do a leave one out cross validation on my data using this code. I have the following ...
7
votes
2answers
204 views

Selection of k knots in regression smoothing spline equivalent to k categorical variables?

I'm working on a predictive cost model where the patient's age (an integer quantity measured in years) is one of the predictor variables. A strong nonlinear relationship between age and risk of a ...
1
vote
2answers
102 views

Strict convexity of Ridge vs Convexity of LASSO

Is there any intuition why the ridge regression is strictly convex, while the LASSO is only convex? Does it have to do with the "corners" of the L1 regularization?
2
votes
0answers
40 views

How can I tell if I my sample size is large enough for reliable feature selection using LASSO regression?

I have a gene expression dataset with 20 samples, and am not going to be getting any more. There are ~28,000 genes and four clinical covariates associated with each sample. The gene expression values ...
3
votes
1answer
247 views

Ridge, lasso and elastic net

How do ridge, LASSO and elasticnet regularization methods compare? What are their respective advantages and disadvantages? Any good technical paper, or lecture notes would be appreciated as well.
1
vote
1answer
98 views

Why no mention of penalized regression techniques in Applied Logistic Regression, 3rd edition, by Hosmer, Lemeshow, and Sturdivant?

Just ordered this textbook, and Wow, the complete omission of this subject from an otherwise excellent reference on logistic regression is a bit surprising. The 2nd edition was published in 2000 - ...
1
vote
0answers
35 views

scikit learn: add lasso or ridge penalty only on subset of parameters

Is there a way of using the linear model api to add the lasso penalty for a subset of the parameters I am regressing? For example, consider a linear separable decomposition that I want to fit to some ...
25
votes
6answers
421 views

Standard errors for lasso prediction using R

I'm trying to use a LASSO model for prediction, and I need to estimate standard errors. Surely someone has already written a package to do this. But as far as I can see, none of the packages on CRAN ...
0
votes
1answer
270 views

Logistic regression using penalized likelihood (lasso?) in Matlab/R

I am trying to use logistic regression in a scenario where there are very few positives. I'm aware that maximum likelihood suffers from small sample bias. So ...
7
votes
1answer
113 views

KKT versus unconstrained formulation of lasso regression

L1 penalized regression (aka lasso) is presented in two formulations. Let the two objective functions be $$ Q_1 = \frac{1}{2}||Y - X\beta||_2^2 \\ Q_2 =\frac{1}{2}||Y - X\beta||_2^2 + \lambda ...
0
votes
1answer
130 views

Superiority of LASSO over forward selection/backward elimination in terms of the cross validation prediction error of the model

I obtained three reduced models from a original full model using forward selection backward elimination L1 penalization technique (LASSO) For the models obtained using forward selection/backward ...
2
votes
1answer
137 views

cv.glmnet lambda stability

Intro I am running cv.glmnet from the glmnet package in R. I am running 10-fold cross ...
1
vote
0answers
48 views

Implementation of generalized cross-validation for constrained LASSO problem

I have a non-negativity constrained LASSO problem like this: min: $||Cx-b||_2^2 + \lambda||x||_1$ subject to: $x\geq0$ where C is a matrix, and x and b are column vectors. Now I want to ...
1
vote
0answers
104 views

Can the predicted value vs observed value plot have a slope not equal 1 in a LASSO model?

I was trying to use glmnet package in R to create a lasso regression model. The details of my data are: Dependent variables $y$: 451 observations, single value for each observation. Independent ...
3
votes
2answers
66 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
3
votes
1answer
88 views

Interpreting the lasso coefficients

I have used lasso logistic regression on some data and I have some non zero coefficients for some of the features. I want to know based upon the coefficient values how do I rank the features?
4
votes
1answer
63 views

Averaging LASSO coefficients for repeated random partitioning of data

Is it reasonable to average LASSO coefficients from repeated reshuffling of training/test sets? Suppose I randomly divide my data into testing & training sets, then within the training set use ...