A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

learn more… | top users | synonyms

2
votes
1answer
68 views

Things that I am not sure about “LASSO” regression method

I have read the chapters that are related to "LASSO" regression in: The elements of statistical learning (Tibshirani et al.) Statistical Learning with Sparsity: The Lasso and Generalizations. (...
3
votes
2answers
47 views

Random forest and LASSO regression both give different variable importances

I have a dataset with 163 observations (all countries in the world with population > 1000000) and 290 variables related to their disease burden and performance. Because there are more variables than ...
0
votes
0answers
14 views

Lasso logistic regression with GLMNET and fixed effects

I have a pretty general question. Suppose one collected data on investments in companies. Further, one wants to find out if some investors are better than others based on investment success (1/0 | ...
2
votes
2answers
35 views

How do Shrinkage Methods change flexibility of a model?

While working through An Introduction to Statistical Learning, I had difficulty clarifying how flexibility relates to Ridge Regression and Lasso. I recognize that both impose penalties on the ...
0
votes
0answers
21 views

How to use LassoCV to obtain most informative independent variables by setting weights of others to zero [closed]

I want to use the lassoCV function from scikit package. In total I've 8000 data. All of these 8000 points have labels. Up to today, I've always used 3 seperated ...
2
votes
1answer
56 views

Challenges in interpretation of variable selection from LASSO and OLS [duplicate]

I work as a consultant and I am often faced with variable selection and prediction problems. For my clients, I run OLS and I am recently pushing for penalized methods which can handle variable ...
-1
votes
1answer
57 views

sparsity using lasso [closed]

MAtlab code X = randn(5,3); r = [0;2;0;]; Y = Xr + randn(5,1).1; B = lasso(X,Y); B(:,25) this is my code and i get the following output 0 1.3937 ...
0
votes
0answers
17 views

glmnet for mixed models?

I perfom a lasso logistic regression using glmnet and want to account for fixed and random effects. I found R packages that can fit mixed models, e.g. glmmLasso and glmmboot. However, is it possible ...
0
votes
0answers
14 views

Can LARS or Coordinate Descent select features that are marginally uncorrelated with the response?

I could construct a response Y the following way: Given $\left\lbrace X_k \right\rbrace_{k=1}^p$, and the regression model $$ Y = \sum_{i = 1}^p X_i - \rho p \beta_{p+1}X_{p+1} + \varepsilon,$$ if ...
-1
votes
1answer
13 views

Help with hierNet package in R [closed]

I have been having some trouble using the hierNet package in R to run a logistic LASSO. I keep receiving this error message after I try to run this line, ...
0
votes
0answers
36 views

Update rule for multi-task lasso used in scikit-learn

The document of scikit-learn said it use coordinate descent for training multi-task lasso. I have tried to derive the update rule but its too hard for me. Can you show me what is update rule for multi-...
1
vote
0answers
13 views

Computational complexity of the lasso (lars vs coordinate descent)

The lasso can be computed with the LARS or Coordinate Descent algorithm. What is their computational complexity and when one is quicker than the other?
3
votes
0answers
14 views

LASSO and compatibility constant

I am new on this web-site and coming from the field of economics (although interested in High Dimensional Statistics), I am reading Statistics for High Dimensional Data of Bühlmann and Van De Geer. I ...
0
votes
0answers
14 views

Quick question on performance of lasso logisitc model

I performed a lasso logistic regression on two modles. One model contains only the control variables. The other model contains controls+linguistic measures. When I search for the optimum lambda ...
1
vote
1answer
19 views

Squared IV in lasso logistic regression

I use lasso logistic regression and want to test the influence of a suqared (X1^2) variable on the outcome. Assume, I put all variables in my model and it looks sth. like this: Y (0/1) X1 X1^2 X2 ...
2
votes
1answer
20 views

Extension to SAFE screening rule for Lasso

In El Ghaoui et al. (2010), "Safe feature elimination in sparse learning" and following works, screening rules are derived for Lasso (as well as other L1-penalized problems): $ \min_w \|y-X w\|^2 + \...
1
vote
1answer
33 views

Interpret and compare lasso models

I'm using lasso logistic regression in order to identify important variables and make inferences. For that I deploy glmnet and repeated cross validation to identify the best tuning parameters lambda. ...
0
votes
0answers
30 views

Multivariate Elastic Net

For my thesis I want to use a multivariate elastic net. So I've got multiple dependent variables, which I want to estimate simultaneously with Elastic Net. Sort of like the idea of SUR. SUR stands ...
0
votes
1answer
32 views

Lasso logistic cross validated error

I fitted a lasso logistic regression using glmnet. I use a pretty small dataset with only 51 (28/23) observations. I want to compare the model fit of two possible variable combinations. Only ...
0
votes
1answer
22 views

lasso coefficients larger than 1

I am comparing lasso implementation in liblinear and glmnet and I can see one of the coefficients from liblinear fit is larger than 1 , while for the same data set, glmnet coefficients are all ...
1
vote
1answer
26 views

Study design using multinomial vs logistic regression?

Suppose that I have a categorical response variable that consists of group 1-3, and I hope to see if predictors can differentiate group 1 vs group 3 (group 2 not included). The response variable is ...
1
vote
1answer
19 views

Difference between trace plot and cv.glmnet

I am using glmnet to perform lasso logisitc regression. The picture shows the zoomed trace plot to illustrate when the first coefficeints pop out, when relaxing lambda. One can see that this happens ...
0
votes
0answers
11 views

When to use Group lasso over lasso?

Two cases: When should numerous numerical predictors be grouped? is it just based off some theoretical knowledge on the predictors? When should levels(>2) in a factor be grouped together?
4
votes
0answers
81 views

optimism-corrected regression coefficients using Frank Harrell's method?

I used a regularized (LASSO) cox regression to estimate relapse times of patients and used Frank Harrell's bootstrapping method to obtain an optimism-corrected performance estimate of my model. I am ...
3
votes
2answers
80 views

AICc is picking overly complex models - something stricter?

I'd like to know if there are stricter alternatives to automated model selection than AICc / AIC / BIC. We have approximately ten thousand curves, and for each we'd like to find the most parsimonious ...
1
vote
0answers
38 views

Why use group lasso instead of lasso?

I have read the that the group lasso is used for variable selection and sparsity in a group of variables. I want to know the intuition behind this claim. Why is group lasso preferred to lasso? Why ...
2
votes
0answers
33 views

When does LASSO fail? [duplicate]

When does LASSO fail in terms of: out of sample forecasting? selecting relevant variables? (If there is a distinction between 1 and 2 to begin with.) Also, Under what conditions would one be ...
2
votes
1answer
76 views

Lasso will not remove correlated variables

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
1
vote
0answers
40 views

Double lasso variable selection

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso ...
0
votes
2answers
124 views

Lasso for “cherry picking”

I use Lasso logistic regression in order to identify a smaller subset of important variables. I start with N=51 (28/23) and 32 predictors. So far it looks pretty promising, because i can identify ...
-1
votes
2answers
65 views

lasso regression on top of random forest

Some time ago, I found a paper describing usage of lasso/elastic net regression on binary variables come from random forest. In short, (i,j)-th variable takes 1 if given observation belong to leaf no. ...
0
votes
1answer
62 views

Why would LASSO not shrink irrelevant features to zero?

Assume I have 10 features to predict an outcome and I use LASSO regression. Let's say the RMSE of the test set is 20. Now, I introduce 5 more features and predict the same outcome, and I also use ...
0
votes
1answer
39 views

Use Lasso Logistic Regression to Analyze Binary Data with

I am involved with a medical research that analyzes Coronary Artery Disease. The dataset has a couple of predictors such as age, gender, race, certain symptons and medical standard procedures to be ...
0
votes
1answer
15 views

constructing random effects design matrices for lassop{MMS}

I'd like to use elastic net regression for coefficient estimate and parameter selection on a data set that includes nested structure. I've been experimenting with lassop{MMS} to do so. I'm not a ...
0
votes
0answers
11 views

Does the variance inflation factor make sense for regularized regression?

I have a logistic regression model fit using L1 regularization. There are two variables that entered the model that have a correlation of over 0.90. The VIF for these variables are each about 60 which ...
3
votes
1answer
333 views

Ridge/Lasso Lambda greater than 1

I ran Ridge and Lasso regressions using an algorithm to automatically find the optimum lambda. However, the algorithm couldn't find an optimum lambda between 0 and 1. In some cases I could find ...
3
votes
1answer
66 views

Cross validation after LASSO in complex survey data

I am trying to do model selection on some candidate predictors using LASSO with a continuous outcome. The goal is to select the optimal model with the best prediction performance, which usually can be ...
4
votes
1answer
112 views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
1
vote
0answers
43 views

Plogit: Lasso logistic regression with Stata

I want to execute a Lasso logistic regression with Stata. Why? Because... 1.) I have a dummy dependant variable (=> Investment success (1) and failure (0)); samples(1/0)(28/23). 2.) I have a set of ...
0
votes
1answer
12 views

Supplying the number of lambdas in cv.glmnet

The cv.glmnet function states 2 options for its lambda parameter. First is NULL, and then <...
3
votes
1answer
100 views

How is L1 regulaziation derived?

I understand the basic idea of regularization. I am very curious to know the derivations behind it so that I get the complete picture. I though a good place to start leraning about regularization is ...
0
votes
0answers
12 views

convergence of coordinate descent applied to lasso

When using coordinate descent for solving a lasso regression, does normalizing the features impact the convergence rate?
0
votes
0answers
26 views

Interpreting results from lasso regression?

I have a time series data set with about 2million observations and 31 variables, which I break to a few thousand using threshold value for my dependent variable. I am using lasso regression in R to ...
1
vote
0answers
56 views

Autoencoders & Predictive sparse decomposition (PSD) & Alternating Direction Multiplier Method (ADMM)

I am studying Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville. In Chapter #14 Autoencoders the authors write Internally, it has a hidden layer $h$ that describes a code used to ...
2
votes
2answers
62 views

Lasso and Ridge tuning parameter scope

In ridge and lasso linear regression, an important step is to choose the tuning parameter lambda, often I use grid search on log scale from -6->4, it works well on ridge, but on lasso, should I take ...
0
votes
0answers
7 views

Multiple variables / continuous outcome / model formula

I have a set of continuous / discrete variables with which I want to model a continuous outcome. How can I know which type of curve would be a good choice and, therefore, which kind of function to ...
0
votes
0answers
8 views

Iterating Lasso

Can Lasso regression be performed multiple times to systematically clean/remove parameters from a model? Would there be downsides to doing so/would it be considered poor practice? Thanks!
0
votes
0answers
13 views

Unbalanced binary features in LASSO regression

I have a target $y$ that I want to predict from variables $x_1, x_2, \ldots x_k$. Suppose the first of these variables, $x_1$, is a binary variable (i.e., only taking on one of 2 values). If I use ...
0
votes
0answers
9 views

Lasso eliminating features but then they reappear?

Does there exist a set of samples and outputs such that running lasso with some value $\lambda$ for the $\mathcal{L}_1$ penalty zeros out some coefficient $w_i$ but for some $\lambda' > \lambda$, $|...