Questions tagged [elastic-net]

A regularization method for regression models that combines the penalties of lasso and of ridge regression.

303 questions
Filter by
Sorted by
Tagged with
40k views

Ridge, lasso and elastic net

How do ridge, LASSO and elasticnet regularization methods compare? What are their respective advantages and disadvantages? Any good technical paper, or lecture notes would be appreciated as well.
• 5,162
43k views

What is elastic net regularization, and how does it solve the drawbacks of Ridge ($L^2$) and Lasso ($L^1$)?

Is elastic net regularization always preferred to Lasso & Ridge since it seems to solve the drawbacks of these methods? What is the intuition and what is the math behind elastic net?
• 5,603
24k views

Why is lambda "within one standard error from the minimum" is a recommended value for lambda in an elastic net regression?

I understand what role lambda plays in an elastic-net regression. And I can understand why one would select lambda.min, the value of lambda that minimizes cross validated error. My question is Where ...
• 413
3k views

Why does glmnet use "naive" elastic net from the Zou & Hastie original paper?

The original elastic net paper Zou & Hastie (2005) Regularization and variable selection via the elastic net introduced elastic net loss function for linear regression (here I assume all ...
• 105k
49k views

Choosing optimal alpha in elastic net logistic regression

I'm performing an elastic-net logistic regression on a health care dataset using the glmnet package in R by selecting lambda values over a grid of $\alpha$ from 0 ...
• 6,104
23k views

Feature selection & model with glmnet on Methylation data (p>>N)

I would like to use GLM and Elastic Net to select those relevant features + build a linear regression model (i.e., both prediction and understanding, so it would be better to be left with relatively ...
• 565
6k views

Fitting an ARIMAX model with regularization or penalization (e.g. with the lasso, elastic net, or ridge regression)

I use the auto.arima() function in the forecast package to fit ARMAX models with a variety of covariates. However, I often have a large number of variables to select from and usually end up with a ...
• 23.8k
5k views

Bridge penalty vs. Elastic Net regularization

Some penalty functions and approximations are well studied, such as the LASSO ($L_1$) and the Ridge ($L_2$) and how these compare in regression. I've been reading about the Bridge penalty, which is ...
• 19.1k
17k views

Why Lasso or ElasticNet perform better than Ridge when the features are correlated

I have a set of 150 features, and many of them are highly correlated with each other. My goal is to predict the value of a discrete variable, whose range is 1-8. My sample size is 550, and I am using ...
• 857
5k views

Using regularization when doing statistical inference

I know about the benefits of regularization when building predictive models (bias vs. variance, preventing overfitting). But, I'm wondering if it is a good idea to also do regularization (lasso, ...
• 421
24k views

LASSO and adaptive LASSO are two different things, right? (To me, the penalties look different, but I'm just checking whether I miss something.) When you generally speak about elastic net, is the ...
28k views

Is R-squared value appropriate for comparing models?

I'm trying to identify the best model to predict the prices of automobiles, using the prices and features available on automobile classified advertisement sites. For this I used a couple of models ...
• 321
4k views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
• 321
3k views

Is there a Bayesian interpretation of linear regression with simultaneous L1 and L2 regularization (aka elastic net)?

It's well known that linear regression with an $l^2$ penalty is equivalent to finding the MAP estimate given a Gaussian prior on the coefficients. Similarly, using an $l^1$ penalty is equivalent to ...
4k views

Elastic/ridge/lasso analysis, what then?

I'm getting really interested in the elastic net procedure for predictor shrinkage/selection. It seems very powerful. But from the scientific point of view I don't know well what to do once I got the ...
• 2,917
12k views

Why can't ridge regression provide better interpretability than LASSO?

I already have an idea about pros and cons of ridge regression and the LASSO. For the LASSO, L1 penalty term will yield a sparse coefficient vector, which can be viewed as a feature selection method. ...
• 330
1k views

Is there a clear set of conditions under which lasso, ridge, or elastic net solution paths are monotone?

The question What to conclude from this lasso plot (glmnet) demonstrates solution paths for the lasso estimator that are not monotonic. That is, some of the cofficients grow in absolute value before ...
• 12.6k
3k views

Showing the Equivalence Between the ${L}_{2}$ Norm Regularized Regression and ${L}_{2}$ Norm Constrained Regression Using KKT

According to the references Book 1, Book 2 and paper. It has been mentioned that there is an equivalence between the regularized regression (Ridge, LASSO and Elastic Net) and their constraint ...
• 2,089
9k views

Difference between ElasticNet in scikit-learn Python and Glmnet in R

Has anybody tried to verify whether fitting an Elastic Net model with ElasticNet in scikit-learn in Python and glmnet in R on ...
• 954
7k views

Confusion related to elastic net

I was reading this article related to elastic net. They say that they use elastic net because if we just use Lasso it tends to select only one predictor among the predictors that are highly correlated....
• 1,501
7k views

Coordinate descent for the lasso or elastic net

Are there any good papers or books dealing with the use of coordinate descent for L1 (lasso) and/or elastic net regularization for linear regression problems?
• 5,591
7k views

How is tol used in scikit-learn's Lasso and ElasticNet?

As a followup to this question, how does scikit-learn implementation of Lasso (and coordinate_descent algorithm) uses the tol parameter in practice? More precisely, ...
• 253
1k views

Why l2 norm squared but l1 norm not squared?

In the Lasso, and ElasticNet, we use, as penalty, the l1 norm without squaring. But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, ...
1k views

Why is Elastic Net called Elastic Net?

What is the etymology of "Elastic Net" in Elastic Net Regularization? Does it have anything to do with the name of "lasso"? Related: Why is ridge regression called "ridge&...
• 987
8k views

Any disadvantages of elastic net over lasso?

What are the disadvantages of using elastic net in comparison to lasso. I know that the elastic net is able to select groups of variables when they are highly correlated. It doesn't have the ...
• 857
6k views

How is the intercept computed in GLMnet?

I've been implementing the GLMNET version of elastic net for linear regression with another software than R. I compared my results with the R function glmnet in lasso mode on diabetes data. The ...
• 331
1k views

Replicating results for glmnet linear regression using a generic optimizer

As the title states, I'm trying to replicate the results from glmnet linear using the LBFGS optimizer from library lbfgs. This optimizer allows us to add an L1 ...
• 773
906 views

What are some of the most important "early papers" on Regularization methods?

In several answers I have seen CrossValidated users suggest OP find early papers on Lasso, Ridge, and Elastic Net. For posterity, what are the seminal works on Lasso, Ridge, and Elastic Net?
3k views

How to interpret coefficients of a multinomial elastic net (glmnet) regression

I'm trying to model a membership in one of three well-being clusters (flourisher, normative, languisher) based on a set of predictors, using elastic net for both variable selection & modelling. I ...
• 649
1k views

Penalized methods for categorical data: combining levels in a factor

Penalized models can be used to estimate models where the number of parameters is equal to or even greater than the sample size. This situation can arise in log-linear models of large sparse tables of ...
• 3,117
9k views

Cross validation with two parameters: elastic net case

I want to know the cross validation procedure to find the two parameters of elastic net presented by Zou and Hastie on the prostate dataset as example. I can't improve the error rate lasso with k-fold ...
• 1,541
4k views

Stepwise regression vs. elastic net

I understand that Stepwise regression analysis has lots of limitations, including the assumption that the predictors are not highly correlated with each other. In fact, this limitation was the most ...
• 361
2k views

Why does glmnet use coordinate descent for Ridge regression?

If I understand it correctly, glmnet uses cyclical coordinate descent not only for lasso and elastics nets, but also for Ridge regression. Why does it use this algorithm, which sometimes gives ...
4k views

Group elastic net

The lasso and the elastic net are not able to handle variables with more than two categories and therefore a split of categorical variables into dummies is necessary for the application of these ...
• 1,218
2k views

Range of lambda in elastic net regression

$\def\l{|\!|}$ Given the elastic net regression $$\min_b \frac{1}{2}\l y - Xb \l^2 + \alpha\lambda \l b\l_2^2 + (1 - \alpha) \lambda \l b\l_1$$ how can an appropriate range of $\lambda$ be chosen ...
• 3,682
142 views

I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms 1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$ 2) $\exp\{-\frac{(\... • 273 9 votes 0 answers 2k views Selecting regularization penalty: cross validation or information criteria? I will use an elastic net to estimate a regression model which will later be used for forecasting. I have a grid of$\alpha$values within [0,1] representing the proportion of$L_1$versus$L_2$... • 67.5k 8 votes 1 answer 2k views Grid fineness and overfitting when tuning$\lambda\$ in LASSO, ridge, elastic net

I wonder about the optimal grid fineness and what the relation between grid fineness and overfitting is in regularization methods such as LASSO, ridge regression or elastic net. Suppose I want ...
• 67.5k
6k views

Is feature selection with dummy coding of categorical variables problematic? [duplicate]

In the context of feature selection it is common to recode categorical variables with more than 2 categories into dummies. Selection methods such as elastic nets or lasso regression select the best ...
• 1,218
2k views

• 105k