Questions tagged [elastic-net]
A regularization method for regression models that combines the penalties of lasso and of ridge regression.
220
questions
1
vote
0answers
35 views
Sparse PCA for $p >> n$ solution with Elastic Net
I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper.
To derive the sparse axis, ...
2
votes
0answers
27 views
test error lower than training error for several algorithms
I just read this question but all the answers are focused on why this is happening when using a neural network.
I'm using random forest, elastic net and Cubist. Both elastic net and Cubist have lower ...
1
vote
0answers
12 views
How to chose Lamba and Fraction in an elastic net model implemented with caret train?
I have this model :
...
2
votes
0answers
47 views
How is `tol` used ins scikit-learn's `Lasso` and `ElasticNet`?
As a followup to this question, how does scikit-learn implementation of Lasso (and coordinate_descent algorithm) uses the tol parameter in practice?
More precisely,...
0
votes
1answer
20 views
comparing two regression coefficients obtained by same elastic net regression model
I have 10 responses and 20 predictors for which I measured values in two conditions.
I ran elastic model for each response at each condition separately.
As a result, I will have two association ...
0
votes
0answers
17 views
Effect of log transformation or standardization of a regressor in the filtering step
We are working with a dataset that has hundreds of biomarkers (many of which are correlated) and often they have many missing values. Our initial goal was to use an elastic net but that would require ...
3
votes
0answers
26 views
How to consider interaction terms in the ridge / lasso / elastic net?
I would like to ask a question about how to consider interaction terms in my penalized regression? My primary goal is to build the model to predict. I think in the conventional GLM, we run the model ...
1
vote
1answer
53 views
Significance test for coefficients of elastic net
I have a 150x41 dataset, on which I performed variable selection and regression with Elastic Net. The response variable is continuous. I'd like to test the significance of the coefficients that I ...
1
vote
0answers
29 views
Getting ElasticNet weight estimates
Regarding Zou and Hastie (2005), they formulate an augmented problem with $(X^*, y^*, \lambda_1, \lambda_2)$, which they they say can be used in a lasso-type problem where $\alpha = \frac{\lambda_1}{\...
1
vote
1answer
54 views
Poor performance on Regularized models
I'm trying to build a simple model to predict the price of a cab ride, using features
such as hour, source, destination, car model, distance, and
weather features such as pressure and humidity.
I've ...
2
votes
1answer
22 views
Validating the way of creating a composite score using elastic net
We have a dataset where we want to see the association of different personal/demographic characteristics and biomarkers with post-injury depression. For that, we first want to create a composite score ...
1
vote
0answers
11 views
Elastic Net: diverging number of parameters
I am reading the paper On The Adaptive Elastic-Net With a Diverging Number of Parameters by Zou and Zhang (2009). I found it while I was researching the lasso and elastic net in general and I am ...
6
votes
1answer
518 views
Why is Elastic Net called Elastic Net?
What is the etymology of "Elastic Net" in Elastic Net Regularization? Does it have anything to do with the name of "lasso"?
1
vote
1answer
27 views
Elastic Net Regression in R formula
This is not a question of how to do Elastic Net Regression in R, but understanding the objective function of Elastic Net Regression from the glmnet package.
From the package itself the objective ...
3
votes
1answer
115 views
Biased prediction (overestimation) for xgboost
I run xgboost and elastic-net on the same dataset for a classification problem, say we have
...
5
votes
1answer
202 views
How to interpret coefficients of a multinomial elastic net (glmnet) regression
I'm trying to model a membership in one of three well-being clusters (flourisher, normative, languisher) based on a set of predictors, using elastic net for both variable selection & modelling. I ...
2
votes
1answer
39 views
Feature Engineering for Multiple Regression
My goal:
I want to predict a single output value from multiple inputs, some of which are numerical and some of which are categorical. To do this, I plan on building a multiple-regression model (...
2
votes
1answer
219 views
L1 and L2 regularization showing increased MSE with added vars (that eventually decreases)
I am attempting to run Ridge, LASSO, and Elastic Net regression as the regularization approaches are commonly used in the problem I'm working to solve.
I have successfully run both glmnet() and cv....
0
votes
0answers
18 views
Number of Variables in Elastic Net
I have a data set with 1000 observations and 150 independent variables. When I apply elastic net, I end up with 100 variables. I wonder if I need to do any additional feature selection or if I can use ...
0
votes
0answers
11 views
elastic net with random interventions
Since elastic net gets rid of regressors that do not correlate, would it make sense to randomly put in a bunch of interventions (pulses, level shifts, ect)?
If the intervention is not needed, won't ...
0
votes
0answers
13 views
Does elastic net with Arima errors make sense?
I know of regression with arima errors, but can one also do elastic net regression with arima errors?
I ask because I read somewhere that the residuals for elastic net are not really valid since ...
2
votes
1answer
45 views
Elastic net/LASSO with soft labels
Sometimes you do not have firm Y/N labels, but e.g. 80% probability of Y as a label. E.g. this happens, if you train a model on a small amount of labelled data, predict for a large amount of ...
0
votes
0answers
40 views
Elastic net regression with uneven penalties for predictors
For a regression model where you are certain that y that depends on some predictors but are agnostic about whether some other predictors should enter, how should you incorporate this prior information?...
8
votes
1answer
81 views
What is the correct way to write the elastic net?
I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms
1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$
2) $\exp\{-\frac{(\...
0
votes
0answers
70 views
How train function in caret choose lamda for elastic net
I'm a beginner in elastic net. I'm using following code for elastic net in R
...
0
votes
1answer
133 views
Why is Lasso and Ridge not giving better results than OLS?
I am trying to find an example in which Lasso and Ridge regression are doing better than simple OLS.
I am trying to run the Boston example that appears in the MASS library in R. The dependent ...
0
votes
0answers
32 views
Ridge vs. Lasso vs. Elastic Net [duplicate]
I have a theoretical question. I was reading about ridge regression, lasso and the elastic net, and is very impressed. One thing is not quite clear to me.
I would like to know when should I use each ...
1
vote
0answers
37 views
How does regularized regression overcome the p > n problem?
So, I understand why simple linear or logistic regression will have infinite solutions in this case (good answers here and here). But while LASSO will only select n features, Elastic net does not have ...
2
votes
0answers
86 views
Outliers and influential observations in elastic net logistic regression
My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
0
votes
0answers
57 views
Elastic Net - number of non-zero variables
I have a question regarding the interpretation of the trace of coefficients when running Elastic net with the package glmnet in R.
This is the plot I obtain with alpha = 0.5
My understanding is that ...
5
votes
2answers
260 views
One-to-one correspondence between penalty parameters of equivalent formulations of penalised regression methods
Ridge, LASSO and Elastic Net are three very popular methods of penalised regressions. All of these have more than one formulations. For example, two formulations for Ridge are:
minimise $\lVert Y - X ...
1
vote
0answers
51 views
Why does `sklearn.linear_model.enet_path` give different results than `glmnet_python`?
The results alpha path from enet_path and the lambda path from ...
0
votes
0answers
36 views
Elastic net chooses lags beyond ACF cutoff
I've been using Elastic net for time series forecasting. I’m using first difference of the series. Normally I use the ACF to determine the number of lags to use. I was curious, if I would produce more ...
0
votes
0answers
70 views
Regularization techniques (ridge, lasso or elastic-net regression) for meta-analysis
I was perusing the StatQuest YouTube channel, and serendipitously listened to a series of videos on regularization techniques (ridge, lasso or elastic-net regression).
It came to my mind that such ...
1
vote
2answers
782 views
How to interpret / metric Lasso regression coefficients
Edited Question, since it was a duplicate
I used Matlab to make a lasso model for my data that has 41 predictors and 1 response variable, and perhaps I used more variables that I need too or maybe ...
1
vote
0answers
25 views
R glmnet package: why is elasticnet parametrized like that? [duplicate]
The documentation for glmnet in R here states that it solves the following minimization problem:
Why did the authors choose to ...
11
votes
2answers
629 views
Showing the Equivalence Between the $ {L}_{2} $ Norm Regularized Regression and $ {L}_{2} $ Norm Constrained Regression Using KKT
According to the references Book 1, Book 2 and paper.
It has been mentioned that there is an equivalence between the regularized regression (Ridge, LASSO and Elastic Net) and their constraint ...
0
votes
1answer
167 views
CrossValidation in ElasticNet Lambda Parameter
Generally, 10-fold CV is used to find the best $\lambda$ (the shrinkage parameter, not the trade-off between L1 and L2 norm $\alpha$ - see https://stats.stackexchange.com/a/64278/185237) in an elastic ...
0
votes
0answers
12 views
Help: Null Values for penalized SVM
I am interested in fitting a SVM model to my data with Elastic SCAD penalty. I was trying to use the penalizedSVM library for this. The issue is that for some reason, the library outputs a null model ...
1
vote
0answers
32 views
Ridge, Lasso or Elastic nets used in Accounting Research
I am trying to come up with ideas for my master's thesis and was wondering why literature on the above mentioned regression methods within Accounting Research is non-existent?
I felt like the ...
2
votes
0answers
56 views
Probability that feature selection in elastic net regularisation is meaningful - evaluating the statistical significance of chosen features
I have a research question - can I use baseline clinical features to predict my binary clinical outcome in individual patients? I am interested if the performance of my model is greater than chance. I ...
0
votes
0answers
69 views
Cross-validation with one-standard-rule for Elastic Net
I would like to use one-standard -deviation rule for my hyper-parameter selection for elastic net. I understand how to do it for ridge or lasso, but when it comes to two regularization parameters I am ...
1
vote
1answer
38 views
How to check if covariates in multiple regression is explaining the same?
I am a master's student doing my thesis at the moment and have come to the point of determining my empirical setup. I would like to get some guidance, in terms of what I am thinking is proper..
I ...
0
votes
1answer
562 views
Interpreting glmnet cox coefficients
There have been similar questions regarding interpretation of glmnet results. However this is more specific to the cox part of the package.
I am trying to create a prognostic score for cancer ...
1
vote
1answer
228 views
Is there any two-stage procedure for elastic net as LASSO?
I read this post Why use Lasso estimates over OLS estimates on the Lasso-identified subset of variables?
. It says the LASSO shrinkage causes the estimates of the non-zero coefficients to be biased ...
1
vote
1answer
106 views
why does lasso select at most n predictors?
From the seminal paper on elastic net regularization from Zou and Hastie 2005, I read
...
2
votes
1answer
360 views
R squared / deviance explained for elastic net glmnet
I am using R glmnet function for the elastic net for logistic regression with binary outcome and would like to calculate the R-square value.
I am getting different results when I use the dev.ratio ...
0
votes
1answer
1k views
how to use elastic net to select a set of features
I have a dataset with 500 samples and 100 features. I need to come up with a set of features. The management prefer a model with a smaller set of features. How exactly should I use elastic net to do ...
2
votes
0answers
54 views
The limitations of Elastic-net regularization [duplicate]
I know that Elastic-net regularization is the combination of L1 and L2 regularization.
My question is what are the limitations of Elastic-net regularization?
My question is not related to Elastic-...
5
votes
0answers
32 views
Equations of Elastic net regularization [duplicate]
I know that the Elastic net takes care of the limitations of Lasso by adding an L 2 penalty term. In the attached picture has been mentioned that the two formulas are equivalent. I tried to show this, ...
