Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

Filter by
Sorted by
Tagged with
0
votes
1answer
15 views

LASSO regression coefficient values different from regular glm

So I have done a LASSO regression, and the output is about 5 coefficients and an intercept. However, if I choose the same variables (I know I shouldn't do that) as the LASSO regression find, and input ...
4
votes
0answers
18 views

Statistical Learning With Sparsity: Direct Inspection of the LASSO function

In the book Statistical Learning with Sparsity: The Lasso and Generalizations, in section 2.4.1, they mention that the absolute value of $\beta$ has no derivative at $\beta=0$, therefore they proceed ...
1
vote
1answer
25 views

Additional variable reduction after LASSO?

I have done some variable reduction before my LASSO regression. I end up with 13 variables (from 100). Some of them are kind of similar. Like for example, if I wanted to predict if it were going to ...
0
votes
0answers
46 views

Lasso regression doesn't converge in case of zero Y-vector

I try to use lasso regression to solve linear problem with big amount of equations (~10 000). Everything worked fine, but I noticed that if in Y-vector all elements are equal, "fit" function hang for ...
0
votes
0answers
21 views

Can I use LASSO as a variable selector with only three variable?

Can we still use LASSO even when we have a small number of variables?
6
votes
1answer
408 views

Why is Elastic Net called Elastic Net?

What is the etymology of "Elastic Net" in Elastic Net Regularization? Does it have anything to do with the name of "lasso"?
2
votes
0answers
34 views

If $\ell_0$ regularization can be done via the proximal operator, why are people still using LASSO?

I have just learned that a general framework in constrained optimization is called "proximal gradient optimization". It is interesting that the $\ell_0$ "norm" is also associated with a proximal ...
0
votes
0answers
5 views

Variance Inflation Factor on many variables

I have a good amount of variables, where some/most of them are highly correlated. Like roughly 80 variables, and then maybe 50 are correlated in groups. So 20 of the 50 are highly correlated, and the ...
0
votes
0answers
14 views

Do adaptive lasso, group exponential lasso and composite MCP methods have the same optimization algorithm of the likelihood function?

As part of survival analysis and variable selection, I would like to compare adaptive lasso, group exponential lasso (gel) and composite MCP (cMCP) methods. In terms of cross-validation, do these ...
0
votes
0answers
8 views

How to do cross validation in ODE models with more predicted than measured time courses?

I have an ODE model of biochemical reactions with 37 state variables and 88 strictly positive parameters. Unfortunately, I can only expect to get time course measurements of about 10 state variables (...
1
vote
0answers
7 views

Non-differentiable regression penalties having non-zero probability of inducing sparsity

I was wondering if anyone has any analytic insight into why non-differentiable penalties can set coefficients to zero, and if there is a more relaxed alternative to this definition regarding the ...
0
votes
0answers
7 views

What is the prior of $\ell_{2,1}$ loss in Multi-Task learning?

We all know Laplacian prior is the prior for Lasso, as the MAP of a Bayesian setting. Multi-task lasso is a generalized lasso for multi-task problems, which encourages group-wise sparisty. However, ...
1
vote
0answers
27 views

Lasso: more penalization with more data?

I am currently doing a backtest of a financial data set with an expanding window. For this, I estimate a Lasso model each month. Hence, each month that I estimate the model, I will have more data. ...
1
vote
1answer
27 views

Model evaluation for feature selection

I have a dataset of gene expression data and I'm trying to find genes related to particular diseases. My labels are dichotomous (sick - not sick) and I used a Logistic regression with LASSO ...
0
votes
0answers
16 views

What's the rationale behind multiple response LASSO?

I understand that, with LASSO, the regularization term puts a constraint on the complexity of our regression model. Usually, for prediction applications, regularization makes the model perform better ...
1
vote
0answers
19 views

How to pick the model that minimizes the mean absolute error when the amount of observations is small

I am given a data set with 1 target variable and 12 features for only 18 observations. My goal is to build a model that has the smallest expected prediction error. I am allowed to use simple methods ...
2
votes
2answers
32 views

Combining results of multiple Lasso runs / Variable selection

I would appreciate your opinion on an analysis approach I have in mind. The idea is to do the variable selection with multiple runs of Lasso regression (by glmnet in R). Basically, the workflow would ...
1
vote
1answer
28 views

Conceptual problem with logistic regression using LASSO regularization and cross-validation

When performing k-fold cross-validation (CV) then k training sets are used to generate k models. This is not a problem to understand when each model contains the same predictors. The conceptual ...
3
votes
1answer
39 views

Why glmnet 's $\lambda$ value is so small? Does it strictly implement the loss function under the hood?

I am running a glmnet fit with 1200000 samples. According to the glmnet doc, $\lambda$ value is the coefficient controlling how much the regularization term contributes to the total loss function. ...
0
votes
1answer
32 views

Get odds ratios with confidence intervals from a lasso regression model

I try to understand lasso regression. So far, I do understand that it can be used to shrink regression coefficients in case of few events. The coefficients of some covariates are even shrunk to zero. ...
0
votes
0answers
21 views

Estimated coefficients after SAS glmselect with Lasso

I did run a lasso logistic regression with SAS glmselect (Y=1 for event and Y=-1 for non event). My syntax is something like: ...
0
votes
1answer
23 views

heteroscedasticity evaluation of residuals in linear LASSO regression model [closed]

I plotted residuals for linear LASSO model. Though tests for heteroscedasticity doesn't show any but i am seeing one some lines in residual plots depicting some heteroscedasticity might be present. I ...
0
votes
0answers
31 views

LASSO Regression with noise

I know LASSO regression is useful to exclude redundant features, so can it be useful when you have noisy data? I explain better with this example: Suppose I generated a data set using an equation (e....
0
votes
1answer
15 views

Selected variables varies depending on whether or not standardization is in lasso regression (glmnet)

The paper often suggests both standardized and unstandardized coefficients in the lasso model (glmnet in R). However, when I run glmnet, the selected variable is different depending on standardized =...
0
votes
0answers
10 views

Working Response variable and Weights for Logistic Regression (glmnet)

I am trying to figure out how the intercept is calculated for logistic regression lasso using coordinate descent algorithm based on this seminal paper: https://www.ncbi.nlm.nih.gov/pmc/articles/...
2
votes
0answers
39 views

Checking non linear effects in LASSO regression

This might be a weird question and I understand that LASSO is mainly using as a variable selection method. But I want to know that is it possible to check non-linear effects of a LASSO logistic ...
3
votes
0answers
80 views

Is it correct to use the model provided by LASSO to predict an outcome?

I know my question will sound a bit stupid to the experts of the field but I can't find a good answer to this point. I have 214 covariates and a binary outcome. The total number of positive and ...
0
votes
0answers
47 views

Events per variable in LASSO regression

I'm using LASSO regression to select genetic variables that predict time to an event (using glmnet with Cox in R). I have approximately 300 predictors and 1,000 events. I'd like to keep my events per ...
3
votes
0answers
36 views

Why does LASSO predict random data “well” during leave-one-out cross validation?

pre-amble: While investigating different cross validation strategies for small sample size dataset's with relatively large number of features I came across this peculiar result. While making a simple ...
0
votes
0answers
9 views

Intercept meaning in group lasso

I have a database with categorical and continuous variable. My response variable is dichotomous and the indipendent variables are 4 factors (2 of them with roughly 10 levels and 2 dichotomous) and 20 ...
0
votes
1answer
45 views

What does lars return for lambda equal to zero when p is larger than n?

For the lasso problem $$ \hat \beta(\lambda) = \underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}}\|Y - X \beta\|_2^2 + \lambda \|\beta\|_1, $$ where $Y$ is $p \times 1$, $X$ is $n \times p$, it ...
0
votes
0answers
38 views

Comparing Ridge and Lasso Regression [duplicate]

I was thinking about main differences between ridge and lasso introducing a $\ell^2$ and $\ell^1$ penalty term respectively. The main thing is that with ridge I will keep all my features in the end ...
1
vote
1answer
51 views

Why standardization of design matrix $X$ with factor $\frac{1}{n}$ instead of $\frac{1}{n-1}$ in lasso/glmnet?

I'm a little bit puzzled by the default standardization of the lasso/elastic net/ridge regression algorithms implemented in the (great!) glmnet package. In most other applications, people would ...
0
votes
0answers
22 views

What is the difference between lasso and WOE encoding in logistic regression?

I know lasso is one of the best method to select important variables and make variables sparse. But WOE encoding does the same thing, making variable smooth. I would like to know what is the ...
1
vote
0answers
44 views

Is it possible to calculate LASSO by hand?

I'm using LASSO regression from Sklearn, and I wanted to see if it is possible to calculate the coefficients and predictions by hand? Currently I'm just fitting the model and then using the predict ...
0
votes
0answers
25 views

Equation to update slope vector using lasso regression

I am trying to figure out how sklearn lasso model works. I am using gradient descent process under which I am performing lasso Where I have defined cost function as ...
0
votes
1answer
48 views

Determining Intercept for Regularized Logistic Regression

Going off of the standard set up, we have $N$ observations and $P$ predictors stored in the data matrix $\mathbf{X} = \{ x_{i,j} \}$ for $i = 1, \ldots, N$ and $j = 1, \ldots, P$. The response is ...
3
votes
0answers
31 views

Hyperprior for Bayesian LASSO & Horseshoe

I currently sample LASSO and Horseshoe regression in STAN. Hence I was wondering how to properly define the hyperpriors in the bayesian regression models. I.e. Park and Casella use a gamma prior with ...
0
votes
0answers
59 views

Coding Lasso method in competing risk regression(FG Model)

Suppose, I have approximated my penalty function quadratically. I already have correct algorithm when tuning parameter is zero. It is simply the case when lambda=0. So, to incorporate lasso, I need to ...
2
votes
0answers
25 views

Prediction intervals for Lasso predictions

I am trying to find confidence (prediction) intervals for predictions of new data by a Lasso model. I've fitted a Lasso model to training data using the cv.glmnet function (i.e. by cross-validation) ...
2
votes
1answer
120 views

L1 and L2 regularization showing increased MSE with added vars (that eventually decreases)

I am attempting to run Ridge, LASSO, and Elastic Net regression as the regularization approaches are commonly used in the problem I'm working to solve. I have successfully run both glmnet() and cv....
0
votes
0answers
10 views

Test Error on various types of Regression

I'm testing a dataset for various types of regression, comparing test error for each one to the Mean Prediction Error, that I found at the beginning. Unfortunately I don't have any experience in this ...
1
vote
1answer
55 views

Feature Selection when LASSO sets all coefficients to zero

I am trying to select features for a data set that contains what food people have eaten with a result of BMI. However, since I want to see what foods impact BMI most I am trying to use various forms ...
0
votes
0answers
45 views

Can data ever be too high dimensional for the Lasso?

I'm trying to implement Lasso on high dimensional textual data. Format of Data: p ~= 45,000, n~=4,000 When running the Lasso, I get a training score of 0 and the number of features selected as 0. ...
0
votes
0answers
27 views

Adaptive Lasso:msgps, glmnet, or glmaag?

To perform Adaptive lasso, which is the best package: msgps, or glmnet or glmaag? What's the difference among the three packages: algorithm, results?
0
votes
1answer
64 views

implement lasso Regression in GAMs in R

Hello I search a way to select variables in a gam function R by using Lasso Regression. I already fitted a model gam(y~x) with x ...
0
votes
0answers
41 views

Creating a risk score from Cox Regression

I have two datasets with palliative cancer patients including 106 and 60 patients, respectively. I have biomarkers of inflammation and coagulation, as well as clinical characteristics for all patients....
0
votes
0answers
42 views

Mathematical proof of how L1 and L2 regularization work [duplicate]

How do you mathematically prove that L1 regularization makes weights sparse but L2 regularization does not?
0
votes
1answer
72 views

Cox regression with lasso regression

Is it possible to perform lasso regression (glmnet with "cox") for variable selection and then conduct Cox regression using ...
1
vote
2answers
151 views

Minimum and maximum regularization in L0 (pseudo)norm penalized regression

L0-pseudonorm penalized least squares regression (aka best subset regression) solves $\widehat{\beta}(\lambda)$ as $$\min_\beta \frac{1}{2}||y-X\beta||_2^2 +\lambda||\beta||_0.$$ where $||\beta||_0$ ...