Questions tagged [elastic-net]

A regularization method for regression models that combines the penalties of lasso and of ridge regression.

Filter by
Sorted by
Tagged with
13 votes
3 answers
1k views

Why l2 norm squared but l1 norm not squared?

In the Lasso, and ElasticNet, we use, as penalty, the l1 norm without squaring. But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, ...
1 vote
0 answers
27 views

The importance of stationarity for the oracle property of Elastic Net Regression?

I've been on the lookout for a while, but unfortunately, I'm still coming up empty-handed in my search for papers or books that dive into the theoretical derivation or simulation of the impact of non-...
1 vote
1 answer
280 views

Constructing random effects design matrices for lassop{MMS}

I'd like to use elastic net regression for coefficient estimate and parameter selection on a data set that includes nested structure. I've been experimenting with lassop{MMS} to do so. I'm not a ...
3 votes
1 answer
2k views

How to use weights with Elasticnet regression in python?

I am using Elasticnet from scikit-learn in python, I've also used Glmnet package in R for prototyping. I want to use weights in Elasticnet which apparently is not available as an option/argument in ...
0 votes
0 answers
27 views

How to plot/visualize correlation values from two different methods for comparison?

I am working on a project wherein we are comparing two methods used for modeling gene expression: one method is using elastic net and other is using lasso regression. In one method: we see that ...
4 votes
1 answer
2k views

Soft-thresholding for the LASSO with complex valued data

I'm currently implementing coordinate descent for the LASSO with complex-valued data. For this, one needs a complex version of the soft-thresholding operator, which seems hardly available on the net. ...
2 votes
0 answers
45 views

Variable selection in multiply imputed data

I have a dataset with approximately 1800 observations and I'm trying to fit a multivariable logistic regression model (250 cases, 1550 controls). There are 19 covariates (mix of continuous, ordinal ...
0 votes
0 answers
40 views

How can you constrain the intercept of a glmnet model to be positive?

If I use the lower.limits = 0 argument, it doesn't apply to the intercept for some reason. I can't find any documentation as to why or how to do it. Any ideas? ...
3 votes
0 answers
26 views

For variable selection, would a viable alternative to using lasso be to use ridge with a threshold, or is switching to elastic net preferred?

A similar question was asked here Why can't ridge regression provide better interpretability than LASSO?, and the answer suggested that a main difference between lasso and ridge is that a zero ...
2 votes
1 answer
371 views

Meta-parameter search for elastic net regularization of general objective function

In their 2004 paper on elastic net regularization, Zou and Hastie present an efficient method for finding the meta-parameters by folding the $L_2$-regularization component into the OLS problem and ...
0 votes
0 answers
27 views

weird lasso prediction when using lambda 1se

I have performed a leave-one out cross-validated prediction using a lasso regression (with both lambda min and lambda 1se). My sample size is 52 and I have a bit more than 20 predictors. While lambda ...
0 votes
0 answers
13 views

Statistical analysis to interpret beta effect size for two different elastic net model

I have two elastic net model and I want to compare their coefficient to say if they have any significant beta effect changes across these two models. I thought of using Anova but realized since we don'...
0 votes
0 answers
27 views

Proximal operator of Adaptive Elastic Net

I would like to learn how to find the proximal operator of the Adaptive Elastic Net, from DOI: 10.1214/08-AOS625 "ON THE ADAPTIVE ELASTIC-NET WITH A DIVERGING NUMBER OF PARAMETERS" by HUI ...
0 votes
0 answers
17 views

How to interpret glmnet coefficients when computed on ln-transformed data

So I'm analyzing data from a paper I found, which includes categorical variables like sex and disease severity as well as lipidomics data. I wanted to try using elastic-net regression to find ...
2 votes
1 answer
93 views

How does regularized regression overcome the p > n problem?

So, I understand why simple linear or logistic regression will have infinite solutions in this case (good answers here and here). But while LASSO will only select n features, Elastic net does not have ...
5 votes
0 answers
92 views

Generalize the 1SE rule to elastic net

When you do LASSO or ridge regression, and pick the hyperparameter using cross-validation, the 1SE rule suggest to select not the best CV result but the one with the most penalization that's still ...
0 votes
0 answers
37 views

95% confidence interval for C-index after running elastic net for a cox model, and how to get net reclassification index

Can someone please show me how to get 95% confidence interval for c-index in the elastic net codes below: ...
2 votes
1 answer
118 views

Imbalanced logistic elasticnet regression

I am performing a logistic elastic net regression to assess which variables influence the outcome and evaluate it. I am working with an imbalanced dataset that consists of 50 cases and 1700 controls. ...
0 votes
0 answers
34 views

Which standard error of out-of-sample prediction errors is used to select the model one standard error from the minimum?

In this post I established that the standard error of cross-validation prediction error is the standard deviation of prediction error across folds divided by the square root of the number of folds. ...
0 votes
0 answers
81 views

What is the standard error of average cross validation error in elastic net or lasso?

I don't have any colleagues who I can ask about this so I must turn to my colleagues on Cross Validated. I am fitting a stacked adaptive elastic net regression and am having some trouble understanding ...
2 votes
0 answers
36 views

Coordinate Descent Alternating between LASSO and Ridge

Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient? For example, apply a LASSO constraint to a predefined 3 variables and Ridge to ...
1 vote
2 answers
442 views

Elastic Net: How to get more sparsity than "lambda.1se" in R package glmnet

Package glmnet provides a cross validation function called cv.glmnet that allows us to choose between two suggested models (from the many), labelled "lambda.min" and "lambda.1se". ...
0 votes
0 answers
13 views

Do I have to use the 'one standard error from minimum CV error' rule? [duplicate]

I have just spent a decent amount of time learning how to do a stacked adaptive elastic net regression on several multiply-imputed datasets using the saenet package....
1 vote
0 answers
12 views

Can elasticnet ever select a different set of predictors than LASSO for a given lambda? [closed]

Since ridge regression can never penalize coefficients to zero, can elasticnet ever select a different set of predictors than LASSO for a given lambda?
3 votes
0 answers
111 views

Bias correction for elastic net

Background: I have a gene expression dataset (p=6000) which I am using it to predict weights of participants (n=31), with elastic net in glmnet package in R. To estimate the bias of the model, 10-...
1 vote
0 answers
277 views

Best Datasets and Packages for Comparing LASSO, Elastic Net, and Ridge [closed]

I have been recently been working with the MASS, lars, and glmnet packages to study variable ...
1 vote
0 answers
26 views

Differences in Performance Between in MASS Package lm.ridge() and enet in elasticnet Package

A background: I am currently working with the 'elasticnet' package (elasticnet v.1.3) maintained by Hui Zou. This package was developed to accompany Hui Zou and Trevor Hastie's Statistical Society B ...
15 votes
3 answers
3k views

Showing the Equivalence Between the $ {L}_{2} $ Norm Regularized Regression and $ {L}_{2} $ Norm Constrained Regression Using KKT

According to the references Book 1, Book 2 and paper. It has been mentioned that there is an equivalence between the regularized regression (Ridge, LASSO and Elastic Net) and their constraint ...
1 vote
0 answers
16 views

Can I do a second pass Elastic Net only including the most significant predictors?

First off, I am new to predictive modeling and I appreciate any advice. BACKGROUND: I am doing a binomial elastic net where n = 54 and p = 89. This model is for predicting drug effects clinically; ...
0 votes
0 answers
25 views

What is the upper limit of the modulus of the coefficient calculated by elastic net regression?

If there is a Elastic-net criterion function: $$\mathcal{L}(\boldsymbol{\beta}) = \frac{1}{2}\sum_{n=1}^N(\boldsymbol{\beta}^{\top}\boldsymbol{x}_n - y_n)^2 + \frac{1}{2}\lambda(1-\eta)\|\boldsymbol{\...
0 votes
1 answer
102 views

Nonzero coefficient ordering in least-square LASSO

Consider least-square LASSO over standardized training data $(\boldsymbol{X},\boldsymbol{y})$. Assume $|\boldsymbol{x}_j\cdot\boldsymbol{y}|>|\boldsymbol{x}_k\cdot\boldsymbol{y}|$. In other words, $...
0 votes
0 answers
105 views

Elastic net can be seen as lasso

Let $y \in \Bbb R^n$, $\Bbb 1$ be an n-vector with all its entries equal to $1$, and $Z \in \Bbb R^{n×p}$ with columns of unit norm and such that $Z^T \Bbb 1 = 0$. The elastic net is a penalized ...
0 votes
1 answer
29 views

How to transfer a trained ElasticNet model to a new dataset? Can Lambda and Alpha determine a unique ElasticNet model?

I have trained a ElasticNet model on a A dataset and also I get the two hyperparameters of the trained ElasticNet model Lambda (ratio of Lasso and Ridge) and Alpha (penalty). I want to see the ...
4 votes
2 answers
4k views

Penalized regression with zero-inflated models

I'm currently building zero-inflated Poisson & negative binomial predictive models using the zeroinfl() function from the pscl package in R. Incorporating penalized regressions into my model to ...
7 votes
2 answers
4k views

Elastic net package for mixed effects models?

I know about glmmLasso but would prefer to use elastic net. I wonder if there are any glmm analogues of glmnet out there, or if ...
14 votes
2 answers
7k views

Coordinate descent for the lasso or elastic net

Are there any good papers or books dealing with the use of coordinate descent for L1 (lasso) and/or elastic net regularization for linear regression problems?
0 votes
0 answers
63 views

When should you use L1 vs. L2 regularization? [duplicate]

Can't seem to find a good explanation online of concrete examples of where you would use one over the other? I also read somewhere that L1 is supposedly slower than L2, but not sure how that is since ...
1 vote
1 answer
198 views

Elastic Net Collinearity

When performing linear regression it is often assumed that the predictors are independent with Gaussian noise: \begin{equation} Y = X\beta + \epsilon \quad \epsilon \sim \mathcal{N}(0, \sigma) \end{...
11 votes
2 answers
8k views

Any disadvantages of elastic net over lasso?

What are the disadvantages of using elastic net in comparison to lasso. I know that the elastic net is able to select groups of variables when they are highly correlated. It doesn't have the ...
4 votes
1 answer
1k views

Stability of tuning parameters in repeated elastic-net and variable retrieval

I am working with data that has $p$ close to $n$, and a high degree of collinearity between predictors. The excellent Introduction to Statistical Learning and Elements of Statistical Learning led me ...
0 votes
0 answers
174 views

Binary Classification using Machine Learning Models for longitudinal data in R

So I have longitudinal data with a binary target variable, and I'd like to perform binary classification using a random forest, xgboost, and glmnet (ridge/lasso/elastic net) model. Is this possible to ...
2 votes
1 answer
152 views

logistic regression with independent variable not-normally distributed (but potentially normally distributed)

I have a question about logistic regression. I am trying to make a model to predict 0 or 1 from several continuous and categorical variables. I know that one continuous variable X is normally ...
6 votes
1 answer
1k views

L1 and L2 regularization showing increased MSE with added vars (that eventually decreases)

I am attempting to run Ridge, LASSO, and Elastic Net regression as the regularization approaches are commonly used in the problem I'm working to solve. I have successfully run both glmnet() and cv....
1 vote
1 answer
327 views

Estimating size of validation cohort

We have generated an elastic net model on a small dataset, where we use gene expression data to calculate a biomarker score to discriminate patients with condition X vs controls. The dataset is too ...
6 votes
1 answer
405 views

Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
4 votes
1 answer
205 views

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
1 vote
1 answer
704 views

How to chose Lamba and Fraction in an elastic net model implemented with caret train?

I have this model : ...
3 votes
1 answer
828 views

How to interpret elastic net coefficients for multinomial regression?

I ran a elastic net regression path for a multinomial model using the glmnet package in R. My response variable has 3 levels (0, 1 and 2 for 3 different stages of a ...
2 votes
1 answer
392 views

CrossValidation in ElasticNet Lambda Parameter

Generally, 10-fold CV is used to find the best $\lambda$ (the shrinkage parameter, not the trade-off between L1 and L2 norm $\alpha$ - see https://stats.stackexchange.com/a/64278/185237) in an elastic ...
1 vote
0 answers
88 views

Comparing an elastic net model with a nested linear regression

Suppose I have a linear model $y_i=\mathbf{x}_i\boldsymbol{\beta}+\mathbf{z}_i\boldsymbol{\gamma}+\epsilon_i$, where $\boldsymbol{\gamma}$ is subject to elastic net regularization. Now I have a nested ...

1
2 3 4 5
7