Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

1,001 questions
Filter by
Sorted by
Tagged with
15 views

LASSO regression coefficient values different from regular glm

So I have done a LASSO regression, and the output is about 5 coefficients and an intercept. However, if I choose the same variables (I know I shouldn't do that) as the LASSO regression find, and input ...
18 views

Statistical Learning With Sparsity: Direct Inspection of the LASSO function

In the book Statistical Learning with Sparsity: The Lasso and Generalizations, in section 2.4.1, they mention that the absolute value of $\beta$ has no derivative at $\beta=0$, therefore they proceed ...
25 views

Additional variable reduction after LASSO?

I have done some variable reduction before my LASSO regression. I end up with 13 variables (from 100). Some of them are kind of similar. Like for example, if I wanted to predict if it were going to ...
46 views

Lasso regression doesn't converge in case of zero Y-vector

I try to use lasso regression to solve linear problem with big amount of equations (~10 000). Everything worked fine, but I noticed that if in Y-vector all elements are equal, "fit" function hang for ...
21 views

Can I use LASSO as a variable selector with only three variable?

Can we still use LASSO even when we have a small number of variables?
408 views

Why is Elastic Net called Elastic Net?

What is the etymology of "Elastic Net" in Elastic Net Regularization? Does it have anything to do with the name of "lasso"?
34 views

If $\ell_0$ regularization can be done via the proximal operator, why are people still using LASSO?

I have just learned that a general framework in constrained optimization is called "proximal gradient optimization". It is interesting that the $\ell_0$ "norm" is also associated with a proximal ...
5 views

Variance Inflation Factor on many variables

I have a good amount of variables, where some/most of them are highly correlated. Like roughly 80 variables, and then maybe 50 are correlated in groups. So 20 of the 50 are highly correlated, and the ...
14 views

Do adaptive lasso, group exponential lasso and composite MCP methods have the same optimization algorithm of the likelihood function?

As part of survival analysis and variable selection, I would like to compare adaptive lasso, group exponential lasso (gel) and composite MCP (cMCP) methods. In terms of cross-validation, do these ...
8 views

How to do cross validation in ODE models with more predicted than measured time courses?

I have an ODE model of biochemical reactions with 37 state variables and 88 strictly positive parameters. Unfortunately, I can only expect to get time course measurements of about 10 state variables (...
7 views

Non-differentiable regression penalties having non-zero probability of inducing sparsity

I was wondering if anyone has any analytic insight into why non-differentiable penalties can set coefficients to zero, and if there is a more relaxed alternative to this definition regarding the ...
7 views

What is the prior of $\ell_{2,1}$ loss in Multi-Task learning?

We all know Laplacian prior is the prior for Lasso, as the MAP of a Bayesian setting. Multi-task lasso is a generalized lasso for multi-task problems, which encourages group-wise sparisty. However, ...
27 views

Lasso: more penalization with more data?

I am currently doing a backtest of a financial data set with an expanding window. For this, I estimate a Lasso model each month. Hence, each month that I estimate the model, I will have more data. ...
27 views

Model evaluation for feature selection

I have a dataset of gene expression data and I'm trying to find genes related to particular diseases. My labels are dichotomous (sick - not sick) and I used a Logistic regression with LASSO ...
16 views

What's the rationale behind multiple response LASSO?

I understand that, with LASSO, the regularization term puts a constraint on the complexity of our regression model. Usually, for prediction applications, regularization makes the model perform better ...
19 views

How to pick the model that minimizes the mean absolute error when the amount of observations is small

I am given a data set with 1 target variable and 12 features for only 18 observations. My goal is to build a model that has the smallest expected prediction error. I am allowed to use simple methods ...
32 views

Combining results of multiple Lasso runs / Variable selection

I would appreciate your opinion on an analysis approach I have in mind. The idea is to do the variable selection with multiple runs of Lasso regression (by glmnet in R). Basically, the workflow would ...
28 views

Conceptual problem with logistic regression using LASSO regularization and cross-validation

When performing k-fold cross-validation (CV) then k training sets are used to generate k models. This is not a problem to understand when each model contains the same predictors. The conceptual ...
39 views

Why glmnet 's $\lambda$ value is so small? Does it strictly implement the loss function under the hood?

I am running a glmnet fit with 1200000 samples. According to the glmnet doc, $\lambda$ value is the coefficient controlling how much the regularization term contributes to the total loss function. ...
32 views

Get odds ratios with confidence intervals from a lasso regression model

I try to understand lasso regression. So far, I do understand that it can be used to shrink regression coefficients in case of few events. The coefficients of some covariates are even shrunk to zero. ...
21 views

Estimated coefficients after SAS glmselect with Lasso

I did run a lasso logistic regression with SAS glmselect (Y=1 for event and Y=-1 for non event). My syntax is something like: ...
23 views

heteroscedasticity evaluation of residuals in linear LASSO regression model [closed]

I plotted residuals for linear LASSO model. Though tests for heteroscedasticity doesn't show any but i am seeing one some lines in residual plots depicting some heteroscedasticity might be present. I ...
31 views

LASSO Regression with noise

I know LASSO regression is useful to exclude redundant features, so can it be useful when you have noisy data? I explain better with this example: Suppose I generated a data set using an equation (e....
15 views

Selected variables varies depending on whether or not standardization is in lasso regression (glmnet)

The paper often suggests both standardized and unstandardized coefficients in the lasso model (glmnet in R). However, when I run glmnet, the selected variable is different depending on standardized =...
10 views

Working Response variable and Weights for Logistic Regression (glmnet)

I am trying to figure out how the intercept is calculated for logistic regression lasso using coordinate descent algorithm based on this seminal paper: https://www.ncbi.nlm.nih.gov/pmc/articles/...
39 views

Checking non linear effects in LASSO regression

This might be a weird question and I understand that LASSO is mainly using as a variable selection method. But I want to know that is it possible to check non-linear effects of a LASSO logistic ...
80 views

Is it correct to use the model provided by LASSO to predict an outcome?

I know my question will sound a bit stupid to the experts of the field but I can't find a good answer to this point. I have 214 covariates and a binary outcome. The total number of positive and ...
47 views

Events per variable in LASSO regression

I'm using LASSO regression to select genetic variables that predict time to an event (using glmnet with Cox in R). I have approximately 300 predictors and 1,000 events. I'd like to keep my events per ...
36 views

Why does LASSO predict random data “well” during leave-one-out cross validation?

pre-amble: While investigating different cross validation strategies for small sample size dataset's with relatively large number of features I came across this peculiar result. While making a simple ...
9 views

Intercept meaning in group lasso

I have a database with categorical and continuous variable. My response variable is dichotomous and the indipendent variables are 4 factors (2 of them with roughly 10 levels and 2 dichotomous) and 20 ...
45 views

What does lars return for lambda equal to zero when p is larger than n?

For the lasso problem $$\hat \beta(\lambda) = \underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}}\|Y - X \beta\|_2^2 + \lambda \|\beta\|_1,$$ where $Y$ is $p \times 1$, $X$ is $n \times p$, it ...
38 views

Comparing Ridge and Lasso Regression [duplicate]

I was thinking about main differences between ridge and lasso introducing a $\ell^2$ and $\ell^1$ penalty term respectively. The main thing is that with ridge I will keep all my features in the end ...
51 views

Why standardization of design matrix $X$ with factor $\frac{1}{n}$ instead of $\frac{1}{n-1}$ in lasso/glmnet?

I'm a little bit puzzled by the default standardization of the lasso/elastic net/ridge regression algorithms implemented in the (great!) glmnet package. In most other applications, people would ...
22 views

What is the difference between lasso and WOE encoding in logistic regression?

I know lasso is one of the best method to select important variables and make variables sparse. But WOE encoding does the same thing, making variable smooth. I would like to know what is the ...
44 views

Is it possible to calculate LASSO by hand?

I'm using LASSO regression from Sklearn, and I wanted to see if it is possible to calculate the coefficients and predictions by hand? Currently I'm just fitting the model and then using the predict ...
25 views

Equation to update slope vector using lasso regression

I am trying to figure out how sklearn lasso model works. I am using gradient descent process under which I am performing lasso Where I have defined cost function as ...
48 views

Determining Intercept for Regularized Logistic Regression

Going off of the standard set up, we have $N$ observations and $P$ predictors stored in the data matrix $\mathbf{X} = \{ x_{i,j} \}$ for $i = 1, \ldots, N$ and $j = 1, \ldots, P$. The response is ...
31 views

Hyperprior for Bayesian LASSO & Horseshoe

I currently sample LASSO and Horseshoe regression in STAN. Hence I was wondering how to properly define the hyperpriors in the bayesian regression models. I.e. Park and Casella use a gamma prior with ...
59 views

Coding Lasso method in competing risk regression(FG Model)

Suppose, I have approximated my penalty function quadratically. I already have correct algorithm when tuning parameter is zero. It is simply the case when lambda=0. So, to incorporate lasso, I need to ...
25 views

Prediction intervals for Lasso predictions

I am trying to find confidence (prediction) intervals for predictions of new data by a Lasso model. I've fitted a Lasso model to training data using the cv.glmnet function (i.e. by cross-validation) ...
120 views

L1 and L2 regularization showing increased MSE with added vars (that eventually decreases)

I am attempting to run Ridge, LASSO, and Elastic Net regression as the regularization approaches are commonly used in the problem I'm working to solve. I have successfully run both glmnet() and cv....
10 views

Test Error on various types of Regression

I'm testing a dataset for various types of regression, comparing test error for each one to the Mean Prediction Error, that I found at the beginning. Unfortunately I don't have any experience in this ...
55 views

Feature Selection when LASSO sets all coefficients to zero

I am trying to select features for a data set that contains what food people have eaten with a result of BMI. However, since I want to see what foods impact BMI most I am trying to use various forms ...
45 views

Can data ever be too high dimensional for the Lasso?

I'm trying to implement Lasso on high dimensional textual data. Format of Data: p ~= 45,000, n~=4,000 When running the Lasso, I get a training score of 0 and the number of features selected as 0. ...
27 views

Adaptive Lasso：msgps, glmnet, or glmaag?

To perform Adaptive lasso, which is the best package: msgps, or glmnet or glmaag? What's the difference among the three packages: algorithm, results?
64 views

implement lasso Regression in GAMs in R

Hello I search a way to select variables in a gam function R by using Lasso Regression. I already fitted a model gam(y~x) with x ...
41 views

Creating a risk score from Cox Regression

I have two datasets with palliative cancer patients including 106 and 60 patients, respectively. I have biomarkers of inflammation and coagulation, as well as clinical characteristics for all patients....
42 views

Mathematical proof of how L1 and L2 regularization work [duplicate]

How do you mathematically prove that L1 regularization makes weights sparse but L2 regularization does not?
L0-pseudonorm penalized least squares regression (aka best subset regression) solves $\widehat{\beta}(\lambda)$ as $$\min_\beta \frac{1}{2}||y-X\beta||_2^2 +\lambda||\beta||_0.$$ where $||\beta||_0$ ...