Questions tagged [elastic-net]

A regularization method for regression models that combines the penalties of lasso and of ridge regression.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Negative Adjusted R-Squared, as evaluation Metric in Ridge, Lasso & Elastic Net Regression

In my R script I'm trying to find the best regression model, in a problem with 7 explanatory variables and my yield~target variable. I have 80 rows of data & the split i'm currently using is 75% ...
2
votes
2answers
267 views

How can a glmnet model with no coefficients have perfect performance?

I sometimes run into situations where glmnet appears to be performing well but actually selects zero features. The AUC is near-perfect but the ...
0
votes
0answers
2 views

Grouping correlated noise features in elastic net

In one online course on machine learning the lecturer said that the main advantage of elastic net penalty is the grouping effect. But also he said that this effect is not ideal because all correlated ...
0
votes
0answers
15 views

Make cv.glmnet select something between lambda.min and lambda.1se [migrated]

I'm training an Elastic Net model and am finding that lambda.1se is much higher than lambda.min, often the maximum lambda tested ...
1
vote
2answers
29 views

How does repeated cross validation go about 'averaging' model coefficients?

No matter how much I google, I cannot find the answer to this simple question. Say you do 10-fold, repeated (5x) CV logistic regression with elastic net regularization. For alpha you try ...
0
votes
0answers
6 views

LASSO/ Elastic Net without test set

I have a small data set (N = 200, 9 predictors, 1 continuous outcome variable) with a lot of noise. I am not able to get "more" data. I want to achieve variable selection. If I split up the data set ...
1
vote
0answers
21 views

Do we need to do multicollinearity check if we are building a lasso/ridge/Elastic net regression models?

I have built an elastic net model for classification purpose, but I haven't done multi-collinearity check. Would doing multicollinearity check and then feeding the variables to the model have an ...
1
vote
0answers
28 views

Do random forests offer advantages over elastic net regression for variable selection?

I have a dataset with 13 predictors and 330 observations (if need be, this can be combined with a second data set which we originally meant to use as replication data for a total of 550 observations). ...
1
vote
0answers
16 views

Should an elastic-net always outperform lasso?

Since the lasso is a subset of the elastic-net, shouldn't a continuous grid of the ridge and lasso paramaters in the elastic-net always outperform the lasso regression?
1
vote
0answers
28 views

Effect size after using elastic net and selective inference

I've employed Elastic net to fit a logistic model with predictors that displayed high degrees of correlation between themselves. I wanted to be able to see which predictors significantly influenced ...
0
votes
0answers
54 views

Extremely high intercept from Elastic Net

I am using the ElasticNet library from sklearn. I am using one predictor which takes values in the range [827.559, 827.5625]. When I fit the model using this ...
0
votes
0answers
13 views

In sklearn MultitaskElasticNet, how should the alpha and l1_ratio scale with number of dimensions?

It seems the norms use have no normalization for number of dimensions in feature or target space. Is there some naive 1/dim scaling that we should use? if you run a fit with a very wide problem and ...
1
vote
0answers
33 views

Regularised Regression and Feature Scaling

When performing regularised regression, such as LASSO, ridge regression and elastic net, I understand that it is important to scale variables before calculating and applying a penalty term. I have ...
2
votes
0answers
23 views

Understanding scaling coefficients in the Naive Elastic Net

So I was reading the original Elastic Net paper by Zou, Hastie and I got slightly confused in the second section, where the reduction from Elastic Net to Lasso is performed. They propose that an ...
0
votes
0answers
30 views

Elastic Net scikit-learn: same model and input data but different prediction values

I was training an Elastic Net using scikit-learn and I bumped into the following problem. I am getting different prediction values for the same input data and model. What is happening? Am I missing ...
0
votes
0answers
8 views

What’s the difference between PQL regression and ridge or lasso or elastic net?

What’s the difference between the penalized methods, glmmPQL and elastic or ridge or elastic net?
1
vote
0answers
50 views

Sparse PCA for $p >> n$ solution with Elastic Net

I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper. To derive the sparse axis, ...
2
votes
0answers
33 views

test error lower than training error for several algorithms

I just read this question but all the answers are focused on why this is happening when using a neural network. I'm using random forest, elastic net and Cubist. Both elastic net and Cubist have lower ...
1
vote
0answers
40 views
3
votes
0answers
398 views

How is `tol` used in scikit-learn's `Lasso` and `ElasticNet`?

As a followup to this question, how does scikit-learn implementation of Lasso (and coordinate_descent algorithm) uses the tol parameter in practice? More precisely, ...
0
votes
1answer
26 views

comparing two regression coefficients obtained by same elastic net regression model

I have 10 responses and 20 predictors for which I measured values in two conditions. I ran elastic model for each response at each condition separately. As a result, I will have two association ...
1
vote
0answers
26 views

Effect of log transformation or standardization of a regressor in the filtering step

We are working with a dataset that has hundreds of biomarkers (many of which are correlated) and often they have many missing values. Our initial goal was to use an elastic net but that would require ...
3
votes
0answers
77 views

How to consider interaction terms in the ridge / lasso / elastic net?

I would like to ask a question about how to consider interaction terms in my penalized regression? My primary goal is to build the model to predict. I think in the conventional GLM, we run the model ...
1
vote
1answer
114 views

Significance test for coefficients of elastic net

I have a 150x41 dataset, on which I performed variable selection and regression with Elastic Net. The response variable is continuous. I'd like to test the significance of the coefficients that I ...
1
vote
0answers
36 views

Getting ElasticNet weight estimates

Regarding Zou and Hastie (2005), they formulate an augmented problem with $(X^*, y^*, \lambda_1, \lambda_2)$, which they they say can be used in a lasso-type problem where $\alpha = \frac{\lambda_1}{\...
1
vote
1answer
104 views

Poor performance on Regularized models

I'm trying to build a simple model to predict the price of a cab ride, using features such as hour, source, destination, car model, distance, and weather features such as pressure and humidity. I've ...
2
votes
1answer
25 views

Validating the way of creating a composite score using elastic net

We have a dataset where we want to see the association of different personal/demographic characteristics and biomarkers with post-injury depression. For that, we first want to create a composite score ...
1
vote
0answers
17 views

Elastic Net: diverging number of parameters

I am reading the paper On The Adaptive Elastic-Net With a Diverging Number of Parameters by Zou and Zhang (2009). I found it while I was researching the lasso and elastic net in general and I am ...
6
votes
1answer
562 views

Why is Elastic Net called Elastic Net?

What is the etymology of "Elastic Net" in Elastic Net Regularization? Does it have anything to do with the name of "lasso"?
1
vote
1answer
36 views

Elastic Net Regression in R formula

This is not a question of how to do Elastic Net Regression in R, but understanding the objective function of Elastic Net Regression from the glmnet package. From the package itself the objective ...
4
votes
1answer
192 views

Biased prediction (overestimation) for xgboost

I run xgboost and elastic-net on the same dataset for a classification problem, say we have ...
5
votes
1answer
438 views

How to interpret coefficients of a multinomial elastic net (glmnet) regression

I'm trying to model a membership in one of three well-being clusters (flourisher, normative, languisher) based on a set of predictors, using elastic net for both variable selection & modelling. I ...
2
votes
1answer
90 views

Feature Engineering for Multiple Regression

My goal: I want to predict a single output value from multiple inputs, some of which are numerical and some of which are categorical. To do this, I plan on building a multiple-regression model (...
5
votes
1answer
318 views

L1 and L2 regularization showing increased MSE with added vars (that eventually decreases)

I am attempting to run Ridge, LASSO, and Elastic Net regression as the regularization approaches are commonly used in the problem I'm working to solve. I have successfully run both glmnet() and cv....
0
votes
0answers
20 views

Number of Variables in Elastic Net

I have a data set with 1000 observations and 150 independent variables. When I apply elastic net, I end up with 100 variables. I wonder if I need to do any additional feature selection or if I can use ...
3
votes
1answer
53 views

Elastic net/LASSO with soft labels

Sometimes you do not have firm Y/N labels, but e.g. 80% probability of Y as a label. E.g. this happens, if you train a model on a small amount of labelled data, predict for a large amount of ...
0
votes
0answers
48 views

Elastic net regression with uneven penalties for predictors

For a regression model where you are certain that y that depends on some predictors but are agnostic about whether some other predictors should enter, how should you incorporate this prior information?...
8
votes
1answer
93 views

What is the correct way to write the elastic net?

I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms 1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$ 2) $\exp\{-\frac{(\...
0
votes
0answers
119 views

How train function in caret choose lamda for elastic net

I'm a beginner in elastic net. I'm using following code for elastic net in R ...
0
votes
1answer
302 views

Why is Lasso and Ridge not giving better results than OLS?

I am trying to find an example in which Lasso and Ridge regression are doing better than simple OLS. I am trying to run the Boston example that appears in the MASS library in R. The dependent ...
0
votes
0answers
44 views

Ridge vs. Lasso vs. Elastic Net [duplicate]

I have a theoretical question. I was reading about ridge regression, lasso and the elastic net, and is very impressed. One thing is not quite clear to me. I would like to know when should I use each ...
1
vote
0answers
41 views

How does regularized regression overcome the p > n problem?

So, I understand why simple linear or logistic regression will have infinite solutions in this case (good answers here and here). But while LASSO will only select n features, Elastic net does not have ...
2
votes
0answers
111 views

Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
0
votes
0answers
69 views

Elastic Net - number of non-zero variables

I have a question regarding the interpretation of the trace of coefficients when running Elastic net with the package glmnet in R. This is the plot I obtain with alpha = 0.5 My understanding is that ...
6
votes
2answers
301 views

One-to-one correspondence between penalty parameters of equivalent formulations of penalised regression methods

Ridge, LASSO and Elastic Net are three very popular methods of penalised regressions. All of these have more than one formulations. For example, two formulations for Ridge are: minimise $\lVert Y - X ...
1
vote
0answers
67 views

Why does `sklearn.linear_model.enet_path` give different results than `glmnet_python`?

The results alpha path from enet_path and the lambda path from ...
0
votes
0answers
37 views

Elastic net chooses lags beyond ACF cutoff

I've been using Elastic net for time series forecasting. I’m using first difference of the series. Normally I use the ACF to determine the number of lags to use. I was curious, if I would produce more ...
1
vote
2answers
1k views

How to interpret / metric Lasso regression coefficients

Edited Question, since it was a duplicate I used Matlab to make a lasso model for my data that has 41 predictors and 1 response variable, and perhaps I used more variables that I need too or maybe ...
1
vote
0answers
26 views

R glmnet package: why is elasticnet parametrized like that? [duplicate]

The documentation for glmnet in R here states that it solves the following minimization problem: Why did the authors choose to ...
13
votes
2answers
834 views

Showing the Equivalence Between the $ {L}_{2} $ Norm Regularized Regression and $ {L}_{2} $ Norm Constrained Regression Using KKT

According to the references Book 1, Book 2 and paper. It has been mentioned that there is an equivalence between the regularized regression (Ridge, LASSO and Elastic Net) and their constraint ...

1
2 3 4 5