Questions tagged [elastic-net]
A regularization method for regression models that combines the penalties of lasso and of ridge regression.
296
questions
0
votes
0
answers
12
views
95% confidence interval for C-index after running elastic net for a cox model, and how to get net reclassification index
Can someone please show me how to get 95% confidence interval for c-index in the elastic net codes below:
...
2
votes
1
answer
50
views
Imbalanced logistic elasticnet regression
I am performing a logistic elastic net regression to assess which variables influence the outcome and evaluate it. I am working with an imbalanced dataset that consists of 50 cases and 1700 controls. ...
- 21
0
votes
0
answers
11
views
Which standard error of out-of-sample prediction errors is used to select the model one standard error from the minimum?
In this post I established that the standard error of cross-validation prediction error is the standard deviation of prediction error across folds divided by the square root of the number of folds. ...
- 1,935
0
votes
0
answers
28
views
What is the standard error of average cross validation error in elastic net or lasso?
I don't have any colleagues who I can ask about this so I must turn to my colleagues on Cross Validated.
I am fitting a stacked adaptive elastic net regression and am having some trouble understanding ...
- 1,935
2
votes
0
answers
23
views
Coordinate Descent Alternating between LASSO and Ridge
Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient?
For example, apply a LASSO constraint to a predefined 3 variables and Ridge to ...
- 1,502
0
votes
0
answers
13
views
Do I have to use the 'one standard error from minimum CV error' rule? [duplicate]
I have just spent a decent amount of time learning how to do a stacked adaptive elastic net regression on several multiply-imputed datasets using the saenet package....
- 1,935
1
vote
0
answers
11
views
Can elasticnet ever select a different set of predictors than LASSO for a given lambda? [closed]
Since ridge regression can never penalize coefficients to zero, can elasticnet ever select a different set of predictors than LASSO for a given lambda?
- 2,076
1
vote
0
answers
71
views
Best Datasets and Packages for Comparing LASSO, Elastic Net, and Ridge
I have been recently been working with the MASS, lars, and glmnet packages to study variable ...
1
vote
0
answers
23
views
Differences in Performance Between in MASS Package lm.ridge() and enet in elasticnet Package
A background:
I am currently working with the 'elasticnet' package (elasticnet v.1.3) maintained by Hui Zou. This package was developed to accompany Hui Zou and Trevor Hastie's Statistical Society B ...
1
vote
0
answers
15
views
Can I do a second pass Elastic Net only including the most significant predictors?
First off, I am new to predictive modeling and I appreciate any advice.
BACKGROUND:
I am doing a binomial elastic net where n = 54 and p = 89. This model is for predicting drug effects clinically; ...
- 11
5
votes
0
answers
55
views
Generalize the 1SE rule to elastic net
When you do LASSO or ridge regression, and pick the hyperparameter using cross-validation, the 1SE rule suggest to select not the best CV result but the one with the most penalization that's still ...
- 29.3k
0
votes
0
answers
21
views
What is the upper limit of the modulus of the coefficient calculated by elastic net regression?
If there is a Elastic-net criterion function:
$$\mathcal{L}(\boldsymbol{\beta}) = \frac{1}{2}\sum_{n=1}^N(\boldsymbol{\beta}^{\top}\boldsymbol{x}_n - y_n)^2 + \frac{1}{2}\lambda(1-\eta)\|\boldsymbol{\...
- 1
0
votes
1
answer
87
views
Nonzero coefficient ordering in least-square LASSO
Consider least-square LASSO over standardized training data $(\boldsymbol{X},\boldsymbol{y})$. Assume $|\boldsymbol{x}_j\cdot\boldsymbol{y}|>|\boldsymbol{x}_k\cdot\boldsymbol{y}|$. In other words, $...
- 219
0
votes
0
answers
54
views
Can Elastic-Net Regression be Trained to Produce Feature Importance?
I've been working on binary classification tasks using Gradient Boosted Trees and Neural Nets and I'd like to calculate the Global importance of each feature in the set.
My model's produce the ...
- 369
0
votes
0
answers
42
views
Elastic net can be seen as lasso
Let $y \in \Bbb R^n$, $\Bbb 1$ be an n-vector with all its entries equal to $1$, and $Z \in \Bbb R^{n×p}$ with columns of
unit norm and such that $Z^T \Bbb 1 = 0$. The elastic net is a penalized ...
- 245
0
votes
1
answer
27
views
How to transfer a trained ElasticNet model to a new dataset? Can Lambda and Alpha determine a unique ElasticNet model?
I have trained a ElasticNet model on a A dataset and also I get the two hyperparameters of the trained ElasticNet model Lambda (ratio of Lasso and Ridge) and Alpha (penalty). I want to see the ...
0
votes
0
answers
24
views
Interpretation of the y axis of a feature importance with elastic net plot
I am doing feature importance using Elastic Net, and have coded and plotted some things. I have a plot in the end which I am not sure that I understand what the y axis means, it ranges from 0 to ...
- 1
12
votes
2
answers
1k
views
Why l2 norm squared but l1 norm not squared?
In the Lasso, and ElasticNet, we use, as penalty, the l1 norm without squaring. But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, ...
0
votes
0
answers
55
views
When should you use L1 vs. L2 regularization? [duplicate]
Can't seem to find a good explanation online of concrete examples of where you would use one over the other?
I also read somewhere that L1 is supposedly slower than L2, but not sure how that is since ...
- 704
1
vote
1
answer
158
views
Elastic Net Collinearity
When performing linear regression it is often assumed that the predictors are independent with Gaussian noise:
\begin{equation}
Y = X\beta + \epsilon \quad
\epsilon \sim \mathcal{N}(0, \sigma)
\end{...
- 121
0
votes
0
answers
139
views
Binary Classification using Machine Learning Models for longitudinal data in R
So I have longitudinal data with a binary target variable, and I'd like to perform binary classification using a random forest, xgboost, and glmnet (ridge/lasso/elastic net) model. Is this possible to ...
2
votes
1
answer
134
views
logistic regression with independent variable not-normally distributed (but potentially normally distributed)
I have a question about logistic regression. I am trying to make a model to predict 0 or 1 from several continuous and categorical variables. I know that one continuous variable X is normally ...
- 21
1
vote
1
answer
243
views
Estimating size of validation cohort
We have generated an elastic net model on a small dataset, where we use gene expression data to calculate a biomarker score to discriminate patients with condition X vs controls.
The dataset is too ...
- 85
4
votes
1
answer
188
views
In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?
Two formulations of the elastic net regression function
Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
- 71
6
votes
1
answer
284
views
Number of samples in scikit-Learn cost function for Ridge/Lasso regression
I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this:
,
whereas the cost function for e.g. Ridge Regression is ...
- 61
1
vote
0
answers
75
views
Comparing an elastic net model with a nested linear regression
Suppose I have a linear model $y_i=\mathbf{x}_i\boldsymbol{\beta}+\mathbf{z}_i\boldsymbol{\gamma}+\epsilon_i$, where $\boldsymbol{\gamma}$ is subject to elastic net regularization. Now I have a nested ...
- 293
2
votes
1
answer
63
views
How to understand regularised linear models
I'm working on a project using elastic nets for predicting a continuous variable using multiple attributes. I'm struggling to understand some of the underlying theory behind what I'm doing.
I ...
- 31
0
votes
1
answer
281
views
Multiple Imputation for Predictors Only, Excluding Missing Outcome Data
I am working with a dataset containing ~300 predictors and ~3000 observations and building a predictive model using elastic net (and hoping to generalize to an external validation set). While the ...
- 15
4
votes
1
answer
392
views
Bayesian priors associated with regularization penalties
I gather that adding a penalty term to (linear) least squares minimization typically corresponds with choosing some prior for Bayes estimation in the normal linear regression model. A couple questions ...
- 523
1
vote
0
answers
103
views
how do I perform permutation testinging for a prediction model developed within caret package (R)?
I'm fairly new to data science/StackExchange, so please excuse any faux pas
I'm trying to perform permutation testing for a chosen ML algorithm (an elastic-net logistic regression) to derive a p-value....
- 21
0
votes
0
answers
63
views
Choosing single lambda and alpha for prediction on independent dataset (Elastic Net)
I have two separate datasets, one of which I am using for training and testing, and the other I am keeping as an independent dataset. My goal is to ultimately train an optimal regression model with ...
- 335
1
vote
1
answer
384
views
Is there a way to write Elastic Net in expanded matrix form?
I am working through a regression problem for a matrix of data that isn't full rank and has more features than observations. For these reasons, I'd like to use elastic net because of its $L1$ and $L2$ ...
- 63
0
votes
1
answer
102
views
How do I find the optimal values for $\beta$ and $\beta_0$ for sparse linear regression model? Where does the mean of $\lambda$ come into account? [closed]
If someone could point me in the right direction that would be greatly appreciated!
Consider the sparse linear regression model:
$\min_{\beta_{0},\beta} \left \{ \frac{1}{2}\left \| \beta _{0}e + X\...
- 3
1
vote
2
answers
990
views
Why are my elastic net and lasso r-squared measures negative?
I'm using sklearn.linear_model.Lasso and sklearn.linear_model.ElasticNet on a model that includes a constant.
I don't expect a model with a constant to perform worse than the average of the data, ie ...
- 143
6
votes
2
answers
674
views
Math behind applying elastic net penalties to logistic regression
I understand how Ridge / Lasso / Elastic Net regression penalties are applied to the linear regression's cost function, but I am trying to figure out how they are applied to Logistic Regression's ...
- 63
1
vote
0
answers
77
views
Solving coefficient sum constrained elastic net with quadratic objective term
I am looking for an algorithm to solve an equality constrained elastic net.
There are two adaptations I need to make to the standard elastic net. First the objective function includes a quadratic ...
- 11
2
votes
0
answers
52
views
Minimizing an elastic net loss function
Reading through Andrew Ng's cs229 notes he shows here on page 10 how you can minimize a loss function $J\left( \theta \right)$ representing the sum of least squares by taking the gradient with respect ...
- 21
1
vote
0
answers
229
views
lambda scaling in elastic net regression with glmnet vs sklearn
I am trying to get results to agree between glmnet and sklearn elastic net regression for a specific case where I can't normalise the response variable y. I know that for ridge regression (alpha = 0) ...
- 111
4
votes
1
answer
170
views
How can I go from Elastic Net Loss to Scikit-Learn Elastic Net?
I couldn't find a better title, but here's the thing...
I was studying Elastic Net regularization and I found this function:
$$
\text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \...
- 317
0
votes
0
answers
134
views
Bivariate analyses vs. Boruta/random forest for removing irrelevant variables prior to penalized regression
I will be using a penalized logistic regression (elastic net) to select variables and their relative importance for predicting an outcome. The goal is to eventually create a risk prediction model, not ...
- 335
2
votes
0
answers
40
views
How/whether to tune Elastic Net parameters using concentration of risk?
Typically, I see alpha and lambda tuned in elastic net models to minimize cross-validated error. Yet, I have seen a handful of articles by one set of authors where they instead tuned parameters to ...
- 335
0
votes
0
answers
247
views
Interpretation of Elastic net having too low or high value of alpha
Often I found the situation that the elastic model what I fitted has optimal alpha value at 0 or 1. Or not only that situation, but also there some alphas go near to 0 or 1.(ex. 0.1 or 0.9)
My ...
1
vote
1
answer
250
views
How to find out hazard ratio and confidence interval from LASSO cox regression ans plot momogram in R?
I am working on a prognostic model based on time to event endpoint. The training data consists of 800 participants and test data around 400. The number of variables is 21. I was using glmnet package.
...
- 13
1
vote
0
answers
69
views
Which elastic net to use in Python [closed]
I need to apply the findings of the Zhou, Hastings Paper in Python. Therefore I wanted to use the sklearn elasticnetCV. But is it the same as the elastic net estimate or the naive elastic net estimate?...
1
vote
0
answers
52
views
Variables that best discriminate groups based on the glmnet package
I am trying to understand how to interpret the result from the glmnet package. What I ultimately want to find is a set of (influential or important) variables that best discriminates three groups (e.g....
- 117
0
votes
1
answer
325
views
How to Improve an Elastic Net Model?
I have a dataset with $n=1500$ observations and $p=2700$ variables.
I fitted an Elastic Net model with $\alpha=0.4$ and $\lambda=0.1$
I chose the $\lambda$ with cross validation, and the $\alpha$ ...
- 35
0
votes
1
answer
429
views
Naive-elastic net and elastic net variable selection comparison
The elastic net paper (here) introduced the naive-elastic net and elastic net. The coefficient estimates of naive-elastic net are obtained by solving the problem
$$\hat\beta_{naive-enet}=\text{argmin}...
- 283
0
votes
0
answers
33
views
Hyperparameter estimation on gaussian regression using elastic net
I have been tasked with explaining how a set of 50 environmental variables can affect 300 plus measures of anatomical regions. I am thinking of using gaussian regression as it can handle multiple ...
- 368
1
vote
2
answers
1k
views
Including Fixed Effects in a LASSO/Elastic Net regression model (in R)
So this is a question has vaguely been asked before (see 1 and 2) but I have not been able to find a conclusive answer for anywhere.
Essentially I have panel data for 300 US firms between 2012-2020 ...
- 11
0
votes
1
answer
335
views
Is there any example of 2-step predictive model?(i.e. classification model coupled with regression models for each subclass)
I have a large dataset with 10,000+ individuals and many many biological features (>5000). And I want to use these features to build a linear model (e.g. elastic net) to predict their clinical ...
- 13