Questions tagged [elastic-net]
A regularization method for regression models that combines the penalties of lasso and of ridge regression.
258
questions
0
votes
1answer
17 views
Including Fixed Effects in a LASSO/Elastic Net regression model (in R)
So this is a question has vaguely been asked before (see 1 and 2) but I have not been able to find a conclusive answer for anywhere.
Essentially I have panel data for 300 US firms between 2012-2020 ...
0
votes
1answer
28 views
Is there any example of 2-step predictive model?(i.e. classification model coupled with regression models for each subclass)
I have a large dataset with 10,000+ individuals and many many biological features (>5000). And I want to use these features to build a linear model (e.g. elastic net) to predict their clinical ...
0
votes
0answers
11 views
Can I test a model on class-imbalanced data if it was trained on class-balanced data?
Background: Previously, I ran my elastic net model on class-imbalanced data. I found out this is bad practice generally, so I downsampled the data to resolve the class imbalance.
...
0
votes
1answer
36 views
Can I run glm after MI with Elastic-Net non-zeroed coefficients from 'miselect'?
I have data with n = 80 and 10 predictors, and ran MI using MICE, followed by Variable Selection for Multiply Imputed Data using ‘miselect’ and finally have 4 non-zeroed coefficients.
Since ...
1
vote
0answers
15 views
Would a log transformation on my features change the elastic net result
We built the elastic net model on a set of my features and control features.
With that, we did various experiments to discuss the importance of the selected features.
For example, we showed more of my ...
2
votes
1answer
31 views
How is the standard error calculated for the `lambda.1se` output in the cv.glmnet function?
I understand that lambda.1se is the largest lambda that gives MSE within one standard error of the minimum MSE. But how is the standard error calculated exactly.
0
votes
1answer
24 views
All p-values rendered non-significant in Cox model after initializing coefficients from glmnet
I have a set of 200 genes that are split into numerical high and low, encoded as (1/2).
I have set this variable this way for linearity of the model. Also, stratified by cancer and normal cases.
I ...
0
votes
0answers
79 views
Elastic net regression: Should I tune for lambda at the same time as alpha with cv.glmnet?
I am looking for the best $\alpha$ (=ratio between L1 and L2 penalty) and $\lambda$ (=penalty strength) for my elastic net regression model, using the R package ...
0
votes
0answers
51 views
Elastic Net / Lasso as a solution to multiple comparisons and p-hacking? Inferential/Descriptive stats
I have a very large dataset, and I'm trying to find which variable(s) may describe the truth about a certain variable. I've considered just doing OLS on variables that make logical sense. But I've ...
0
votes
2answers
18 views
Multicollinearity and case:predictor ratio problems
I have 444 cases and 60ish predictors that suffer from collinearity. The predictors fall into three categories (vol, thickness and demographics). I would prefer to subdivide my cases into 4 (age) ...
0
votes
1answer
28 views
What is the effect of PCA on the Error?
I am fitting an ElasticNet model using an array of values for alpha and l1_ratio.
I then plot the result of the negative root mean squared error from cross validation in a heatmap, which gives me the ...
0
votes
0answers
13 views
Why does the value of the penalized ridge is divided by 2 in GLMNET? [duplicate]
If you look at GLMNET Vignette, it shows that it solves for the gaussian case:
But why does it divide the value of $\parallel \beta \parallel_2^2$ by 2?
0
votes
0answers
9 views
Optimizing models for paired analysis
I want to train a model to classify pre and post samples for 50 patients. The algorithm needs to discriminate ...
0
votes
0answers
24 views
How should I interpret the case where elasticnet gives zero features
I am using elasticnet for the purpose of determining feature importance. In case it is relevant, this is a high-dimensionality model with $n\ll p$.
I have seen cases before where Lasso, i.e. logistic ...
0
votes
1answer
52 views
Is the use of elastic net for variable selection purposes a form of data dredging?
Is the use of elastic net for variable selection purposes a form of data dredging? I switched from using stepwise regression procedure for variable selection to elastic net, but I actually do not know ...
0
votes
1answer
57 views
The cumulative probability model with ordinalNet, using an elastic net penalty: more coefficients than expected equal to zero
I am trying to fit a cumulative probability model (ordinal logistic regression with 17 categories and 827 observations) with elastic net penalty using the ordinalNet...
0
votes
1answer
61 views
What to report in cross validated elastic net regression?
Let's assume I want to construct a regression model to predict a specific outcome variable but I don't have enough data to do a proper train-test set split (n = 200). I have 7 predictor variables (...
0
votes
0answers
9 views
Different results after updating the code (glmnet function) [duplicate]
I wrote a code performing an elastic net regression with a 10-fold-crossvalidation. the original dataset has 190 variables and 1402 observations. The result were 70 remaining coefficients in the final ...
0
votes
0answers
28 views
Normalize parameter in sklearn Ridge, Lasso, ElasticNet [duplicate]
Is there any risk or disadvantage to set normalize=True when using ridge, lasso or elasticnet or does it only have benefits? And what is the impact on the range of alpha if it is set to True, does it ...
0
votes
1answer
91 views
Counterintuitive coefficients in elastic net logistic regression
In a model run of elastic net logistic regression, I encountered a very counterintuitive coefficient. I have looked into the data, model and script, but, I still cannot seem to wrap my head around the ...
1
vote
1answer
197 views
What really is 'glmnet' when used in caret in R for binary classification?
like lasso and ridge, elastic net can also be used for classification
by using the deviance instead of the residual sum of squares. This
essentially happens automatically in caret if the response ...
2
votes
0answers
22 views
Strong rules for the elastic net
In their paper (here), Tibshirani et al defined the lasso as the solution
to
$$
\text{argmin}_{\boldsymbol{\beta}}\frac{1}{2}\left\Vert \mathbf{y}-\mathbf{X}\boldsymbol{\beta}\right\Vert ^{2}+\lambda\...
0
votes
0answers
29 views
What is the range for coefficients for variables in Elastic Net regression?
Does it always range between 0 and 1?
1
vote
1answer
49 views
Is it correct to evaluate individual drivers with the AUC value?
I have a discussion with my supervisor about the usage of AUC to determine, basically, the importance of three different drivers consisting of multiple variables each. He claims I can look into the ...
0
votes
1answer
27 views
Correlated predictors with different signs in elastic net
I am struggling with the interpretation of my elastic net results and hope someone might be able to help ...
I've done an elastic net regression in R (based on glmnet), with different levels of alpha ...
0
votes
1answer
273 views
How do I compare cv.glmnet models with AIC?
I am using the glmnet package in R, and not(!) the caret package for my binary ElasticNet regression. I have come to the point ...
0
votes
0answers
24 views
How do I correct my variable importance calculations?
I am, currently, looking into the relative importance of the different predictors I received from my cv.glmnet coefficients term. But, it seems like there is ...
0
votes
1answer
89 views
Different outcome of cv.glmnet coefficients
I know, the question has been posted many times, but none of the answers fixed my problem. I still get different results each time I run the cv.glmnet on my data. ...
0
votes
0answers
13 views
How does one determine relative importance of clustered variables/drivers?
I have a binary dataset of infected trees consisting of a lot of independent variables that can be divided into three main groups of drivers. Let's intuitively call them ...
1
vote
1answer
346 views
How do I calculate confidence intervals on an elastic net regression in R
I am performing an elastic net regression on my data n = 34, p = 46
I first built the model using the "caret" package with the cross validation method to set the optimal alpha and lambda ...
0
votes
0answers
19 views
Procedure for elastic net, strange MSE
I have been analyzing some data to perform elastic net and I am a little confused which function should I use to build a model for elastic net.
After performing CV with altering alpha with cv.glmnet() ...
2
votes
0answers
142 views
Elastic net: how to tune for both lambda and alpha? Why is caret better than cv.glmnet?
I'm learning how to use R to fit Elastic Net models using the glmnet package. There appears to be a function, cv.glmnet that ...
3
votes
2answers
373 views
How can a glmnet model with no coefficients have perfect performance?
I sometimes run into situations where glmnet appears to be performing well but actually selects zero features. The AUC is near-perfect but the ...
0
votes
0answers
5 views
Grouping correlated noise features in elastic net
In one online course on machine learning the lecturer said that the main advantage of elastic net penalty is the grouping effect.
But also he said that this effect is not ideal because all correlated ...
1
vote
2answers
79 views
How does repeated cross validation go about 'averaging' model coefficients?
No matter how much I google, I cannot find the answer to this simple question.
Say you do 10-fold, repeated (5x) CV logistic regression with elastic net regularization.
For alpha you try ...
0
votes
0answers
9 views
LASSO/ Elastic Net without test set
I have a small data set (N = 200, 9 predictors, 1 continuous outcome variable) with a lot of noise.
I am not able to get "more" data. I want to achieve variable selection.
If I split up the data set ...
1
vote
0answers
41 views
Do we need to do multicollinearity check if we are building a lasso/ridge/Elastic net regression models?
I have built an elastic net model for classification purpose, but I haven't done multi-collinearity check. Would doing multicollinearity check and then feeding the variables to the model have an ...
1
vote
0answers
147 views
Do random forests offer advantages over elastic net regression for variable selection?
I have a dataset with 13 predictors and 330 observations (if need be, this can be combined with a second data set which we originally meant to use as replication data for a total of 550 observations). ...
1
vote
0answers
25 views
Should an elastic-net always outperform lasso?
Since the lasso is a subset of the elastic-net, shouldn't a continuous grid of the ridge and lasso paramaters in the elastic-net always outperform the lasso regression?
1
vote
0answers
57 views
Effect size after using elastic net and selective inference
I've employed Elastic net to fit a logistic model with predictors that displayed high degrees of correlation between themselves. I wanted to be able to see which predictors significantly influenced ...
0
votes
0answers
74 views
Extremely high intercept from Elastic Net
I am using the ElasticNet library from sklearn. I am using one predictor which takes values in the range [827.559, 827.5625].
When I fit the model using this ...
1
vote
0answers
52 views
Regularised Regression and Feature Scaling
When performing regularised regression, such as LASSO, ridge regression and elastic net, I understand that it is important to scale variables before calculating and applying a penalty term. I have ...
2
votes
0answers
62 views
Understanding scaling coefficients in the Naive Elastic Net
So I was reading the original Elastic Net paper by Zou, Hastie and I got slightly confused in the second section, where the reduction from Elastic Net to Lasso is performed.
They propose that an ...
0
votes
0answers
36 views
Elastic Net scikit-learn: same model and input data but different prediction values
I was training an Elastic Net using scikit-learn and I bumped into the following problem. I am getting different prediction values for the same input data and model. What is happening? Am I missing ...
1
vote
0answers
74 views
Sparse PCA for $p >> n$ solution with Elastic Net
I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper.
To derive the sparse axis, ...
2
votes
0answers
35 views
test error lower than training error for several algorithms
I just read this question but all the answers are focused on why this is happening when using a neural network.
I'm using random forest, elastic net and Cubist. Both elastic net and Cubist have lower ...
1
vote
1answer
106 views
How to chose Lamba and Fraction in an elastic net model implemented with caret train?
I have this model :
...
6
votes
0answers
1k views
How is `tol` used in scikit-learn's `Lasso` and `ElasticNet`?
As a followup to this question, how does scikit-learn implementation of Lasso (and coordinate_descent algorithm) uses the tol parameter in practice?
More precisely, ...
0
votes
1answer
39 views
comparing two regression coefficients obtained by same elastic net regression model
I have 10 responses and 20 predictors for which I measured values in two conditions.
I ran elastic model for each response at each condition separately.
As a result, I will have two association ...
1
vote
0answers
35 views
Effect of log transformation or standardization of a regressor in the filtering step
We are working with a dataset that has hundreds of biomarkers (many of which are correlated) and often they have many missing values. Our initial goal was to use an elastic net but that would require ...
