# Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

1,433 questions
Filter by
Sorted by
Tagged with
9 views

### Betas from precision matrix?

This is really 2 questions. I would like to do a linear multi-regression on a large quantity of data, so large that I cannot really store it (it’s about 1e10 observations across 2500 features). Hence ...
16 views

### How to model suspected piecewise-linear data with a lasso GLM

My data consist ~130 observations. Each observation has several thousand features (including many collinear or otherwise useless features) and a position along a single spatial dimension. Some sets of ...
11 views

### Which standard error of out-of-sample prediction errors is used to select the model one standard error from the minimum?

In this post I established that the standard error of cross-validation prediction error is the standard deviation of prediction error across folds divided by the square root of the number of folds. ...
28 views

### What is the standard error of average cross validation error in elastic net or lasso?

I don't have any colleagues who I can ask about this so I must turn to my colleagues on Cross Validated. I am fitting a stacked adaptive elastic net regression and am having some trouble understanding ...
38 views

For both linear and logistic regression we know that the coefficient vector $\hat\beta$ holds an asymptotic normal distribution, therefore the the distribution of the linear predictor $\hat\theta_i=x^... 1 vote 0 answers 24 views ### Proof of Variable Selection Consistency for LASSO in Zhao & Yu, 2006 I'm going through the proof of Proposition 1 in Zhao & Yu, 2006 (https://www.jmlr.org/papers/volume7/zhao06a/zhao06a.pdf), titled On Model Selection Consistency of LASSO. The proof is in Appendix ... 0 votes 0 answers 8 views ### Lasso Regression - Extracting the penalty term If I run a lasso regression model, and get a series of coefficients, say :$\widehat{\beta _{1}},\widehat{\beta _{2}},...,\widehat{\beta _{p}}$Is it legitimate to say that the value of the penalty ... 0 votes 1 answer 24 views ### Which standardization for lasso and dependent variable What kind of standardization of the predictor matrix is needed for lasso? let y be the response variable is it (x-mean(x)/sd(x)) OR (x-mean(x)/sd(y))? Also does lasso require you to standardize the ... 0 votes 0 answers 32 views ### Lasso coefficients and penalization impact What is the correct interpretation of the coefficients from a lasso regression if you standardize ((x-mean(x))/sd(x) the predictor variables? If you observer 4.2 for variable A and -1.2 for variable B ... 0 votes 0 answers 12 views ### Optimal method for predicting outcome from additive, correlated, and sparse features? Suppose I have many vectors which can take on any of three values, 0, 1, 2. These vectors affect an outcome being predicted, Y. Vectors add together: a vector "A" of the value 2 has twice ... 2 votes 0 answers 23 views ### Coordinate Descent Alternating between LASSO and Ridge Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient? For example, apply a LASSO constraint to a predefined 3 variables and Ridge to ... 0 votes 0 answers 7 views ### Compare LASSO regressions as missing data increases I'm comparing three LASSO-regression models for classifying two patient types. Each model has increasingly complex variables, which are less likely to be available. Consequently, the last model has ... 0 votes 0 answers 26 views ### Lasso for non-stationary time series Does it make sense to use Lasso to find an explanatory variable x to predict my variable y, assuming both y and x are non-stationary? (I'm using both variables as levels, not differences). If I find a ... 0 votes 0 answers 16 views ### LASSO shrinking variables RLM finds important Comparing results from LASSO and robust regression I find that in some cases variables with high magnitude coefficients in robust regresison are shrunk to 0 in LASSO. Any intuition here? Coefficients ... 1 vote 0 answers 74 views ### Treatment of blocking variables in LASSO regression By reviewing the existing relevant questions I could not find the answer to this specific question. I have created blocking variables with the one-hot method (n - 1 binary variables for n categorical ... 0 votes 0 answers 37 views ### What's the effect of doubling the data and copying the data for Lasso Regression and Ridge Regression on standard error? Suppose we have a dataset$X$, where each piece data of$X$is a row vector, and the data generation process satisfies Gaussian-Markov assumption. If we do ridge regression on$Y\sim 2X$, how does the ... 0 votes 0 answers 13 views ### Do I have to use the 'one standard error from minimum CV error' rule? [duplicate] I have just spent a decent amount of time learning how to do a stacked adaptive elastic net regression on several multiply-imputed datasets using the saenet package.... 0 votes 0 answers 40 views ### Quasibinomial GLMM with LASSO regularization in R I am currently assessing drivers of deforestation using a GLM (generalised linear model) with LASSO regularization (using package glmnet in R). As the response variable is % of area deforested I have ... 3 votes 0 answers 39 views ### How to show the existence of global minimizer of Lasso type of objective function? Suppose the objective function to be minimized is $$F(\theta) = \|y - X \theta\|_2^2 + \sum_{i=1}^p \lambda_i |\theta_i|$$ where$\theta$is the independent variable which is feasible in$\mathbb{R}^p$... 1 vote 0 answers 14 views ### Increasing accuracy of prediction I'm working with this data set trying to implement a model to predict the variable normexam. I've used the following models on sklearn, adding dummies for categorical variables, and got the following ... 2 votes 0 answers 54 views ### Why do we need$\gamma>2$in SCAD penalty? The SCAD penalty$p(x | \lambda, \gamma)$from https://myweb.uiowa.edu/pbreheny/7600/s16/notes/2-29.pdf or the paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle ... 1 vote 0 answers 10 views ### Is it necessary to choose predictors for svm or I use all my variables? I have transformed my categorical variables to dummies and I have used the lasso method to decide which variables I choose to do the logistic regression, my question is: for the svm model do I need to ... 1 vote 0 answers 11 views ### MSE for ARIMA and UR in linear models I'm trying to assess the predictive power of some time series models (LASSO, ARIMA, UR) on time series data, but I have a problem. I am conducting simulations with$N$data points,$P$potential ... 0 votes 0 answers 7 views ### Selecting sparse number of predictors each derived from a non-sparse b-spline in time I have observations that I would like to model as responses at stations to slowly time-varying inputs at 80 candidate locations, few (perhaps 5-10?) of which are active and significant. I have ... 1 vote 0 answers 11 views ### Can elasticnet ever select a different set of predictors than LASSO for a given lambda? [closed] Since ridge regression can never penalize coefficients to zero, can elasticnet ever select a different set of predictors than LASSO for a given lambda? 0 votes 0 answers 28 views ### Do we need to apply "multi-testing" corrections for the p-values in a regularized model? Say we are fitting a penalized model, such as a linear regression with lasso regularization. We expect to obtain a model with the most significant covariables. The method starts with many covariables ... 0 votes 0 answers 14 views ### Lasso vs. Ridge Regression [duplicate] The lasso coefficients are the ones that minimize$RSS+\lambda \Sigma_{j=1}^{p} |\beta_j|$whereas the ridge regression coefficients those that minimize$RSS+\lambda \Sigma_{j=1}^{p} \beta_j^2$. I don'... 0 votes 1 answer 44 views ### After selecting variables from lasso regression, is it a good practice to re-run the regression with selected variables? As the subject suggests, after selecting regressors from lasso regression, is it a good practice to re-run the an ordinary linearly regression with selected variables? I just feel like intuitively, ... 0 votes 0 answers 12 views ### Graphical Lasso for estimating words network I have a matrix whose columns are words and rows are different speeches by a person. Therefore, the i,j element of the matrix is the count of occurence of a word in a speech. I would like to estimate ... 1 vote 0 answers 42 views ### Which evaluation metric should I choose? AIC or MSE? I am currently at a total loss, so I hope someone can point me in the right direction regarding my model selection. The situation I want to create a linear model that best forecasts my data. I am ... 1 vote 1 answer 41 views ### Tuning lambda in glmnet mgaussian multitask learning model for optimal support recovery I was using a multivariate gaussian (mgaussian) glmnet model to solve the multitask learning problem below (deconvolution of a ... 1 vote 1 answer 40 views ### Bootstrapped Prediction Interval for Adaptive Lasso I am attempting to calculate a 95% prediction interval from an adaptive lasso model using the glmnet:: package in R. I adapted my method from the Python code in ... 0 votes 1 answer 32 views ### From a computational perspective, how does the lasso regression shrink coefficients to 0? I understand the analytic proof that lasso regularisation tends to shrink coefficients to zero. However, from a practical standpoint, most of those methods are combined with gradient optimisation (... 1 vote 1 answer 106 views ### Different sets of features selected by three different functions in R for running LASSO Regressions despite the same random seed for each The GitHub Repository for this research project has all of the code included in this question. Brief background context: I am just finishing up the work on my part as a coauthor on a research project ... 1 vote 0 answers 25 views ### Fused lasso for image denonising For a given data$y_{i}$, with$i=1, \dots, n$, we consider the following signal approximation: $$\hat{y} = \arg \min_{w}\sum_{i=1}^{n}(y_{i}-w_{i})^{2} + \lambda \sum_{(i,j)\in E}|w_{i} - w_{j}|,$$ ... 1 vote 0 answers 30 views ### L1 vs L2 variance? [closed] Which regularisation method L2 or L1 gives a lower variance?$ f(w) = \sum (\hat{y}_i - y_i)^2 + \sum || \beta ||^2 \rightarrow L2  f(w) = \sum (\hat{y}_i - y_i)^2 + \sum || \beta || \rightarrow L1 ... 1 vote
72 views

### Best Datasets and Packages for Comparing LASSO, Elastic Net, and Ridge

I have been recently been working with the MASS, lars, and glmnet packages to study variable ...
1 vote
23 views

### Differences in Performance Between in MASS Package lm.ridge() and enet in elasticnet Package

A background: I am currently working with the 'elasticnet' package (elasticnet v.1.3) maintained by Hui Zou. This package was developed to accompany Hui Zou and Trevor Hastie's Statistical Society B ...
56 views

I recently came across this study describing the benefits of the author's relaxed adaptive LASSO regression. The author describes a simple algorithm (which appears to be glmnet, effectively), as well ...
1 vote
19 views

### Simulation case for Double machine learning in Partial linear model with lasso where number of covariates is large

I want to conduct a simulation study on double machine learning in Partial linear regression setup as described in Chernozukov's paper for Double machine learning. I want to use lasso and want the ...
1 vote
37 views

### Penalty Factor for Adaptive Group LASSO

So gglasso in R appears to allow the user to implement adaptive group lasso by specifying group-wise penalty factors as you would with a typical adaptive lasso. Typically, when I want to train an ...
21 views

### How is the intercept fit in glmnet when doing logistic regression?

In this question, it's explained how the intercept is fit under normal linear regression. It is given that it is calculated using $$\beta_0 = \bar{y} - \sum_{j=1}^p \hat{\beta}_j \bar{x}_j$$ How is ...
1 vote
154 views

### Getting glmnet to select excatly given number of features?

I am using glmnet for feature selection, given a gaussian dependent variable. Part of my code is like this: ...
18 views

### Is there any theory pertaining to an assumption to would allow the following inequality to hold (constrained LASSO)

Suppose we observe the vector-matrix pair $(y,\mathbf{X})\in\mathbb{R}^n\times\mathbb{R}^{n\times d}$ which is linked by the observation model: \begin{equation} y=\mathbf{X}\theta^*+\epsilon \end{...
19 views

### Should instrumental variables abide by the effect hierarchy principle?

People sometimes include the interactions between their instruments and region/year or the interactions between different instruments in the first stage of a 2SLS regression. I wonder if the effect ...
55 views

### Generalize the 1SE rule to elastic net

When you do LASSO or ridge regression, and pick the hyperparameter using cross-validation, the 1SE rule suggest to select not the best CV result but the one with the most penalization that's still ...
1 vote
78 views

### Interpreting survival analysis plot using genes as predictor

I will show what I'm doing it in R to make sure if I'm doing it correctly This is my dataset which I'm using for analysis ...
217 views

### Is standardization still needed after a LASSO model is fitted?

We know that it's better to standardization the training data (i.e. X_train) before fitting a LASSO model, especially when features are not in the same scale (Ref. Is standardisation before Lasso ...