Skip to main content

Questions tagged [elastic-net]

A regularization method for regression models that combines the penalties of lasso and of ridge regression.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Is there any value in models that have a larger out of sample RMSE than a standard deviation?

I am predicting y values from x values using various regression models, elastic net and partial least squares regression (PLSR). To quantify performance of models we utilize root mean squared error (...
Sir Veza's user avatar
1 vote
0 answers
11 views

Assessing Random Search Cross Validation: Tuning in ElasticNet with Large Feature Sets

I'm working on estimating an ElasticNet model for a large dataframe with over 100,000 variables, resulting in a well overidentified scenario. To tune my model, I've set up a grid of hyperparameters (...
george1994's user avatar
3 votes
1 answer
37 views

What is the boundary curve for $λ_1$ and $λ_2$ that give at least a 0 component in elastic net?

Define the elastic net estimate: $ \hat{\beta}^{\lambda_1, \lambda_2} = \arg \min_{\beta \in \mathbb{R}^p} \left( \frac{1}{2n} \| y - X\beta \|_2^2 + \lambda_1 \ \frac{1}{2} \|\beta \|_2^2 + \lambda_2 ...
george1994's user avatar
0 votes
0 answers
30 views

Can you store the value of the predicted variable (Y) at each fold and then correlate the predicted values with the actual data?

In particular, imagine to have a set of features (X) that I use to predict a continuos variable (Y). Is it possible to use elastic net, in a cross-validation framework, use it to predict the value of ...
sup_use's user avatar
1 vote
0 answers
32 views

The importance of stationarity for the oracle property of Elastic Net Regression?

I've been on the lookout for a while, but unfortunately, I'm still coming up empty-handed in my search for papers or books that dive into the theoretical derivation or simulation of the impact of non-...
Joe94's user avatar
  • 95
0 votes
0 answers
27 views

How to plot/visualize correlation values from two different methods for comparison?

I am working on a project wherein we are comparing two methods used for modeling gene expression: one method is using elastic net and other is using lasso regression. In one method: we see that ...
Rhea Bedi's user avatar
2 votes
0 answers
61 views

Variable selection in multiply imputed data

I have a dataset with approximately 1800 observations and I'm trying to fit a multivariable logistic regression model (250 cases, 1550 controls). There are 19 covariates (mix of continuous, ordinal ...
donm79's user avatar
  • 51
0 votes
0 answers
49 views

How can you constrain the intercept of a glmnet model to be positive?

If I use the lower.limits = 0 argument, it doesn't apply to the intercept for some reason. I can't find any documentation as to why or how to do it. Any ideas? ...
PaulG's user avatar
  • 1,297
3 votes
0 answers
29 views

For variable selection, would a viable alternative to using lasso be to use ridge with a threshold, or is switching to elastic net preferred?

A similar question was asked here Why can't ridge regression provide better interpretability than LASSO?, and the answer suggested that a main difference between lasso and ridge is that a zero ...
another_student's user avatar
0 votes
0 answers
36 views

weird lasso prediction when using lambda 1se

I have performed a leave-one out cross-validated prediction using a lasso regression (with both lambda min and lambda 1se). My sample size is 52 and I have a bit more than 20 predictors. While lambda ...
Simon's user avatar
  • 1
0 votes
0 answers
19 views

Statistical analysis to interpret beta effect size for two different elastic net model

I have two elastic net model and I want to compare their coefficient to say if they have any significant beta effect changes across these two models. I thought of using Anova but realized since we don'...
Rhea Bedi's user avatar
0 votes
0 answers
33 views

Proximal operator of Adaptive Elastic Net

I would like to learn how to find the proximal operator of the Adaptive Elastic Net, from DOI: 10.1214/08-AOS625 "ON THE ADAPTIVE ELASTIC-NET WITH A DIVERGING NUMBER OF PARAMETERS" by HUI ...
SupBro's user avatar
  • 35
0 votes
0 answers
19 views

How to interpret glmnet coefficients when computed on ln-transformed data

So I'm analyzing data from a paper I found, which includes categorical variables like sex and disease severity as well as lipidomics data. I wanted to try using elastic-net regression to find ...
maglorismyspiritanimal's user avatar
0 votes
0 answers
45 views

95% confidence interval for C-index after running elastic net for a cox model, and how to get net reclassification index

Can someone please show me how to get 95% confidence interval for c-index in the elastic net codes below: ...
Jeffery Osei's user avatar
2 votes
1 answer
154 views

Imbalanced logistic elasticnet regression

I am performing a logistic elastic net regression to assess which variables influence the outcome and evaluate it. I am working with an imbalanced dataset that consists of 50 cases and 1700 controls. ...
Xfar's user avatar
  • 21
0 votes
0 answers
38 views

Which standard error of out-of-sample prediction errors is used to select the model one standard error from the minimum?

In this post I established that the standard error of cross-validation prediction error is the standard deviation of prediction error across folds divided by the square root of the number of folds. ...
llewmills's user avatar
  • 2,043
0 votes
0 answers
151 views

What is the standard error of average cross validation error in elastic net or lasso?

I don't have any colleagues who I can ask about this so I must turn to my colleagues on Cross Validated. I am fitting a stacked adaptive elastic net regression and am having some trouble understanding ...
llewmills's user avatar
  • 2,043
2 votes
0 answers
38 views

Coordinate Descent Alternating between LASSO and Ridge

Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient? For example, apply a LASSO constraint to a predefined 3 variables and Ridge to ...
Tylerr's user avatar
  • 1,562
1 vote
0 answers
12 views

Can elasticnet ever select a different set of predictors than LASSO for a given lambda? [closed]

Since ridge regression can never penalize coefficients to zero, can elasticnet ever select a different set of predictors than LASSO for a given lambda?
Michael Webb's user avatar
  • 2,176
1 vote
0 answers
326 views

Best Datasets and Packages for Comparing LASSO, Elastic Net, and Ridge [closed]

I have been recently been working with the MASS, lars, and glmnet packages to study variable ...
YessuhYessuhYessuh's user avatar
1 vote
0 answers
26 views

Differences in Performance Between in MASS Package lm.ridge() and enet in elasticnet Package

A background: I am currently working with the 'elasticnet' package (elasticnet v.1.3) maintained by Hui Zou. This package was developed to accompany Hui Zou and Trevor Hastie's Statistical Society B ...
YessuhYessuhYessuh's user avatar
1 vote
0 answers
16 views

Can I do a second pass Elastic Net only including the most significant predictors?

First off, I am new to predictive modeling and I appreciate any advice. BACKGROUND: I am doing a binomial elastic net where n = 54 and p = 89. This model is for predicting drug effects clinically; ...
Ab2's user avatar
  • 11
6 votes
0 answers
106 views

Generalize the 1SE rule to elastic net

When you do LASSO or ridge regression, and pick the hyperparameter using cross-validation, the 1SE rule suggest to select not the best CV result but the one with the most penalization that's still ...
Björn's user avatar
  • 33.5k
0 votes
0 answers
26 views

What is the upper limit of the modulus of the coefficient calculated by elastic net regression?

If there is a Elastic-net criterion function: $$\mathcal{L}(\boldsymbol{\beta}) = \frac{1}{2}\sum_{n=1}^N(\boldsymbol{\beta}^{\top}\boldsymbol{x}_n - y_n)^2 + \frac{1}{2}\lambda(1-\eta)\|\boldsymbol{\...
fei ma's user avatar
  • 1
0 votes
1 answer
109 views

Nonzero coefficient ordering in least-square LASSO

Consider least-square LASSO over standardized training data $(\boldsymbol{X},\boldsymbol{y})$. Assume $|\boldsymbol{x}_j\cdot\boldsymbol{y}|>|\boldsymbol{x}_k\cdot\boldsymbol{y}|$. In other words, $...
user2961927's user avatar
0 votes
0 answers
129 views

Elastic net can be seen as lasso

Let $y \in \Bbb R^n$, $\Bbb 1$ be an n-vector with all its entries equal to $1$, and $Z \in \Bbb R^{n×p}$ with columns of unit norm and such that $Z^T \Bbb 1 = 0$. The elastic net is a penalized ...
Kilkik's user avatar
  • 445
0 votes
1 answer
33 views

How to transfer a trained ElasticNet model to a new dataset? Can Lambda and Alpha determine a unique ElasticNet model?

I have trained a ElasticNet model on a A dataset and also I get the two hyperparameters of the trained ElasticNet model Lambda (ratio of Lasso and Ridge) and Alpha (penalty). I want to see the ...
Kengo Ito's user avatar
13 votes
3 answers
2k views

Why l2 norm squared but l1 norm not squared?

In the Lasso, and ElasticNet, we use, as penalty, the l1 norm without squaring. But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, ...
William de Vazelhes's user avatar
0 votes
0 answers
70 views

When should you use L1 vs. L2 regularization? [duplicate]

Can't seem to find a good explanation online of concrete examples of where you would use one over the other? I also read somewhere that L1 is supposedly slower than L2, but not sure how that is since ...
Katsu's user avatar
  • 1,011
1 vote
1 answer
209 views

Elastic Net Collinearity

When performing linear regression it is often assumed that the predictors are independent with Gaussian noise: \begin{equation} Y = X\beta + \epsilon \quad \epsilon \sim \mathcal{N}(0, \sigma) \end{...
mattapow's user avatar
  • 121
0 votes
0 answers
190 views

Binary Classification using Machine Learning Models for longitudinal data in R

So I have longitudinal data with a binary target variable, and I'd like to perform binary classification using a random forest, xgboost, and glmnet (ridge/lasso/elastic net) model. Is this possible to ...
Armen Abagyan's user avatar
2 votes
1 answer
168 views

logistic regression with independent variable not-normally distributed (but potentially normally distributed)

I have a question about logistic regression. I am trying to make a model to predict 0 or 1 from several continuous and categorical variables. I know that one continuous variable X is normally ...
AKIRA28's user avatar
  • 45
1 vote
1 answer
353 views

Estimating size of validation cohort

We have generated an elastic net model on a small dataset, where we use gene expression data to calculate a biomarker score to discriminate patients with condition X vs controls. The dataset is too ...
mm523's user avatar
  • 85
4 votes
1 answer
217 views

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
tobmo's user avatar
  • 71
6 votes
1 answer
466 views

Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
Holgerillo's user avatar
1 vote
0 answers
91 views

Comparing an elastic net model with a nested linear regression

Suppose I have a linear model $y_i=\mathbf{x}_i\boldsymbol{\beta}+\mathbf{z}_i\boldsymbol{\gamma}+\epsilon_i$, where $\boldsymbol{\gamma}$ is subject to elastic net regularization. Now I have a nested ...
wdg's user avatar
  • 323
2 votes
1 answer
64 views

How to understand regularised linear models

I'm working on a project using elastic nets for predicting a continuous variable using multiple attributes. I'm struggling to understand some of the underlying theory behind what I'm doing. I ...
b247's user avatar
  • 31
0 votes
1 answer
566 views

Multiple Imputation for Predictors Only, Excluding Missing Outcome Data

I am working with a dataset containing ~300 predictors and ~3000 observations and building a predictive model using elastic net (and hoping to generalize to an external validation set). While the ...
NB3's user avatar
  • 15
4 votes
1 answer
653 views

Bayesian priors associated with regularization penalties

I gather that adding a penalty term to (linear) least squares minimization typically corresponds with choosing some prior for Bayes estimation in the normal linear regression model. A couple questions ...
Golden_Ratio's user avatar
1 vote
0 answers
153 views

how do I perform permutation testinging for a prediction model developed within caret package (R)? [closed]

I'm fairly new to data science/StackExchange, so please excuse any faux pas I'm trying to perform permutation testing for a chosen ML algorithm (an elastic-net logistic regression) to derive a p-value....
Graham's user avatar
  • 29
0 votes
0 answers
76 views

Choosing single lambda and alpha for prediction on independent dataset (Elastic Net)

I have two separate datasets, one of which I am using for training and testing, and the other I am keeping as an independent dataset. My goal is to ultimately train an optimal regression model with ...
pdhami's user avatar
  • 335
1 vote
1 answer
518 views

Is there a way to write Elastic Net in expanded matrix form?

I am working through a regression problem for a matrix of data that isn't full rank and has more features than observations. For these reasons, I'd like to use elastic net because of its $L1$ and $L2$ ...
morepenguins's user avatar
0 votes
1 answer
128 views

How do I find the optimal values for $\beta$ and $\beta_0$ for sparse linear regression model? Where does the mean of $\lambda$ come into account? [closed]

If someone could point me in the right direction that would be greatly appreciated! Consider the sparse linear regression model: $\min_{\beta_{0},\beta} \left \{ \frac{1}{2}\left \| \beta _{0}e + X\...
711's user avatar
  • 3
1 vote
2 answers
1k views

Why are my elastic net and lasso r-squared measures negative?

I'm using sklearn.linear_model.Lasso and sklearn.linear_model.ElasticNet on a model that includes a constant. I don't expect a model with a constant to perform worse than the average of the data, ie ...
John Vandivier's user avatar
6 votes
2 answers
812 views

Math behind applying elastic net penalties to logistic regression

I understand how Ridge / Lasso / Elastic Net regression penalties are applied to the linear regression's cost function, but I am trying to figure out how they are applied to Logistic Regression's ...
Stephen's user avatar
  • 63
1 vote
0 answers
118 views

Solving coefficient sum constrained elastic net with quadratic objective term

I am looking for an algorithm to solve an equality constrained elastic net. There are two adaptations I need to make to the standard elastic net. First the objective function includes a quadratic ...
Impatar's user avatar
  • 11
2 votes
0 answers
68 views

Minimizing an elastic net loss function

Reading through Andrew Ng's cs229 notes he shows here on page 10 how you can minimize a loss function $J\left( \theta \right)$ representing the sum of least squares by taking the gradient with respect ...
Redux's user avatar
  • 21
1 vote
0 answers
295 views

lambda scaling in elastic net regression with glmnet vs sklearn

I am trying to get results to agree between glmnet and sklearn elastic net regression for a specific case where I can't normalise the response variable y. I know that for ridge regression (alpha = 0) ...
cno's user avatar
  • 111
4 votes
1 answer
193 views

How can I go from Elastic Net Loss to Scikit-Learn Elastic Net?

I couldn't find a better title, but here's the thing... I was studying Elastic Net regularization and I found this function: $$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \...
Yuxxxxxx's user avatar
  • 317
0 votes
0 answers
143 views

Bivariate analyses vs. Boruta/random forest for removing irrelevant variables prior to penalized regression

I will be using a penalized logistic regression (elastic net) to select variables and their relative importance for predicting an outcome. The goal is to eventually create a risk prediction model, not ...
Mark's user avatar
  • 335

1
2 3 4 5
7