A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

0
votes
0answers
7 views

Meta-parameter search for elastic net regularization of general objective function

In their 2004 paper on elastic net regularization, Zou and Hastie present an efficient method for finding the meta-parameters by folding the $L_2$-regularization component into the OLS problem and ...
2
votes
1answer
26 views

Least Angle regression coefficient reaches zero after included

In LARS how is it possible that after including a variable it could reach zero again? http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf I understood that it works like: 1) choose ...
1
vote
1answer
56 views

Ridge & LASSO norms

this post folloms this one: Why does ridge estimate become better than OLS by adding a constant to the diagonal? Here is my question: As far as I know, ridge regularization uses a l2-norm (euclidean ...
0
votes
0answers
8 views

Coefficient and confidence interval of lasso selection [migrated]

I conducted a feature selection using lasso method as well as a covariance test using covTest::covTest to retrieve the p.values. I borrow an example from ...
3
votes
2answers
101 views

Quadratic Programming and Lasso

I'm trying to perform a lasso regression, which has following form: Minimize $w$ in $(Y - Xw)'(Y - Xw) + \lambda \;\text{norm}(w,1)$ Given a $\lambda$, I was advised to find the optimal $w$ with ...
0
votes
2answers
36 views

Coordinate descent on objective function with discontinuous derivative

I'm trying to perform a customized nonlinear regression. I'm using the Linex loss function instead of least-squares. I'm doing LASSO-style regularization, so that my objective function has ...
4
votes
1answer
176 views

How to interpret variables that are excluded from or included in the lasso model?

I got from other posts that one cannot attribute 'importance' or 'significance' to predictor variables that enter a lasso model because calculating those variables' p-values or standard deviations is ...
0
votes
1answer
27 views

Is standardizing data necessary for glasso?

I am wondering, if it is necessary to standardize data (mean zero and stddev eq. 1) for glasso. In many papers on glasso this is mentioned to have data with mean=0 and stddev 1, while using covarience ...
0
votes
0answers
14 views

model selection through sparse group lasso

I am trying to get familiar with the package SGL. The reference is http://cran.r-project.org/web/packages/SGL/SGL.pdf. I typed the example in, and tried to get the coefficients of the fitted penalized ...
0
votes
0answers
23 views

Lasso equivalent estimator with differenziable penalty function

The lasso estimator is define as $argmin_{\beta}~ MSE +\lambda \parallel \beta\parallel_1$ I am wondering if there is an alternative penalty function that is $C^\infty$ and that preserves the sparsity ...
1
vote
0answers
42 views

Deviance measure in glmnet package

for my current reseach I'm using the Lasso method via the glmnet package in R on a binomial dependent variable. In glmnet the optimal lambda is found via cross-validation and the resulting models ...
0
votes
0answers
21 views

Least-squares fitting with only optimum features, after Lasso - valid?

Using Lasso reduces the coefficients of features of a model, reducing some to zero, and thereby performing feature selection. The number of features depends on the value of $\alpha$ aka $\lambda$. In ...
0
votes
0answers
30 views

Which R packages offer the foldid (or simliar) parameter for cross-validation of group lasso?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 50 (both continuous and categorical) explanatory variables did not come from the top of my head; ...
1
vote
0answers
114 views

Writing a coordinate descent algorithm for elastic net in SAS

In order to run Lasso and elastic net multiple regressions on my company's SAS server (which doesn't support R), I've been working on a coordinate descent macro for performing least squares ...
0
votes
0answers
37 views

covariate selection for a cox model by Lasso using glmnet

I would like to use model selection through shrinkage (Lasso) using glmnet. So far I did the following: ...
0
votes
1answer
88 views

model selection through shrinkage (Lasso) using glmnet

I would like to use model selection through shrinkage (Lasso) using glmnet. After trying the example of the glmnet manual and tried the procedure with my data. ...
0
votes
0answers
28 views

Connection between Lasso formulations

This question might be dumb, but I noticed that there are two different formulations of the Lasso regression. We know that the Lasso problem is to minimize the objective consisting of the square loss ...
2
votes
0answers
37 views

How to compare variables' coefficients obtained from lasso?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 50 explanatory variables did not come from the top of my head; their choice was based on the ...
2
votes
1answer
99 views

How to ensure that the most appropriate value for lambda is chosen in lasso?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 50 explanatory variables did not come from the top of my head; their choice was based on the ...
0
votes
0answers
20 views

Applications of High-dimensional data analysis techniques in Industry

The high-dimensional data analysis in statistics/biostatistics is booming over recent two decades, and is becoming the frontier of modern statistics research. The penalized regularization techniques, ...
2
votes
1answer
71 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
2
votes
1answer
44 views

How is lasso an M-Estimator?

The definition of an M-estimator is an estimator (from Casella and Berger) of the form $$\hat{\theta}=\min \sum_{i=1}^n \rho(X_i-\theta),$$ where $X_1,X_2, \cdots, X_n$ is the data for some function ...
2
votes
1answer
20 views

Variable selection: Why certain categories are chosen but not others?

I'm doing variable selection using the Lasso. To explain my response variable I have several predictors, both categorical and numerical, but I have problems to explain the process that underlies ...
0
votes
0answers
23 views

Code for $\ell_1$/ $\ell_2$ sparse multivariate regression algorithm

Where can I find the code (Matlab or R) for the block-regularized Lasso problem defined as follows: $\min_{B} \| Y-XB\|_F^2 + \lambda \|B\|_{1,2}$ where $Y$, $X$, and $B$ are matrices. This is a ...
0
votes
0answers
17 views

Apply LASSO Model with nominal target in SAS?

I'm building a classification model with a pool of independent variables (hundreds of them). I'm in the step of variable selection/feature selection. Now I'm trying to figure out if there are any ...
0
votes
1answer
62 views

lasso with missing values and categorical variables

I have a dataset with a lot of missing values and mix of continues and categorical variables. I want to use something like group lasso to do features selection. Probably the output is binary 0,1 and ...
0
votes
0answers
36 views

How to derive this problem with soft-thresholding method?

The problem is defined as $$ \min_{x} \Bigg\{ a{\|x\|}^2+\frac{b}{2}{\|x-c\|}^2 \Bigg\} $$ where $x\in R^{n \times 1}, c \in R^{n \times 1}$ and $a,b$ are scalars. Equations 2.5 to 2.8 of this paper ...
1
vote
2answers
101 views

Why is Lasso regression for high dimensional data better than Stepwise AIC?

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So ...
0
votes
1answer
60 views

What is the practical significance of a linear regression, lasso regression, and ridge regression outputting the same coefficients? [closed]

The lasso and ridge regression are tuned to the same alpha parameter. No matter what I tune the parameter to [0,1], the results of all three regressions are always the same (linear, ridge, lasso), ...
2
votes
0answers
30 views

About the derivation of group Lasso

I've been reading the paper of group lasso, "Model selection and estimation in regression with grouped variables". http://www.stat.washington.edu/courses/stat527/s13/readings/yuanlin07.pdf In page 53 ...
1
vote
2answers
26 views

What is the meaning of regularization path in Lasso or related sparsity problems?

If we select different sparsity levels (i.e. that the $\lambda$ controls), we could obtain different solutions with different sparsity levels. Does it mean the regularization path is how to select the ...
6
votes
1answer
68 views

What's the typical range of possible values for the shrinkage parameter in penalized regression?

In lasso or ridge regression, one has to specify a shrinkage parameter, often called by $\lambda$ or $\alpha$. This value is often chosen via cross validation by checking a bunch of different values ...
1
vote
1answer
49 views

How to Find Adjusted $R^2$ or $R^2$ from Lasso and Ridge regression model

How do I find the adjusted $R^2$ (or $r^2$) from Lasso and Ridge regression? I used the glmnet package. For instance if I have this code so far.... ...
1
vote
1answer
63 views

How do you interpret the parameters obtained from lasso logistic regression when the response is binary?

Are we still able to interpret the parameters in the same manner as we would in ordinary logistic regression? I'm asking this because I'm toying with the german credit data ...
1
vote
1answer
67 views

LASSO or other regularized regression with censored (missing) data

Here is my problem. I am looking at various time series curves. Let's call them total spend aggregated over all customers on various products versus time. At any given time, I want to predict the ...
0
votes
1answer
73 views

How do you pronounce “LASSO”?

Some people pronounce it "LAS-so" and some "las-SOO" (from the rope cowboys throw to catch cows). Which way is "the right one"?
1
vote
0answers
38 views

lmmlasso does not work for p>n

I am using the following example from the R manual: ...
6
votes
1answer
74 views

How to decide which penalty measure to use ? any general guidelines or thumb rules out of textbook

A number of regularization measures are available in literatures, which is kind of confusing to beginners. The classical penalty is ridge by Hoerl & Kennard (1970,Technometrics 12, 55–67). ...
1
vote
0answers
87 views

How to deal with “not applicable” values in categorical variables

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
2
votes
1answer
148 views

large variables and low sample (p > n) problem: ridge , LASSO, PLS, PCR which is most suitable for predictions

I am trying see whether to go for ridge regression, LASSO or principal component regression (PCR) or Partial Least Squares (PLS) in a situation where there are large number of variables / features (p) ...
6
votes
2answers
204 views

AIC, BIC and GCV: what is best for making decision in penalized regression methods?

My general understanding is AIC deals with the trade-off between the goodness of fit of the model and the complexity of the model. $AIC =2k -2ln(L)$ $k$ = number of parameters in the model $L$ = ...
3
votes
1answer
120 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
1
vote
2answers
50 views

CV for LASSO tuning parameter using LARS

If I use the LARS algorithm to fit the LASSO path, is it sufficient to cross-validate using the values of $\lambda$ at each step in LARS or is it better to use a finer grid of $\lambda$ values? I ...
13
votes
2answers
181 views

Bayesian lasso vs ordinary lasso

Different implementation software are available for lasso. I know a lot discussed about bayesian approach vs frequentist approach in different forums. My question is very specific to lasso - What are ...
2
votes
1answer
67 views

LASSO in the data with large number of variables (p) with lower number of samples (n)

I would like to fit LASSO in the following type of data where there are large number of variables (p > n). My y variable is y ...
1
vote
0answers
12 views

Repeated multiple regression for LASSO significance testing

I'm currently working on a statistical modelling problem in biology. We have cellular measurements of proteins in every cell in a tissue, and I'm using regression analysis to see if a given protein is ...
1
vote
0answers
30 views

GLS, heteroskedasticity and Ridge Regression/Lasso

I am hoping to use a regularised regression technique, using cross validation, to fit a linear model to a set of predictors which have some highly correlated variables. However, I also know (highly ...
2
votes
0answers
40 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
1
vote
0answers
39 views

Cross-Validation vs. AICc for LASSO

I was working on a research project in which I try to estimate the the individual contribution of a group of regional political leaders to local economic growth. The major challenge is that there is ...
0
votes
0answers
40 views

implementing Lasso with BIC

Do you have an R code to implement Lasso with BIC? Note that there is an R package called ...