Questions tagged [elastic-net]

A regularization method for regression models that combines the penalties of lasso and of ridge regression.

Filter by
Sorted by
Tagged with
3 votes
1 answer
82 views

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
user avatar
  • 31
5 votes
1 answer
55 views

Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
user avatar
1 vote
0 answers
21 views

Comparing an elastic net model with a nested linear regression

Suppose I have a linear model $y_i=\mathbf{x}_i\boldsymbol{\beta}+\mathbf{z}_i\boldsymbol{\gamma}+\epsilon_i$, where $\boldsymbol{\gamma}$ is subject to elastic net regularization. Now I have a nested ...
user avatar
  • 281
2 votes
1 answer
57 views

How to understand regularised linear models

I'm working on a project using elastic nets for predicting a continuous variable using multiple attributes. I'm struggling to understand some of the underlying theory behind what I'm doing. I ...
user avatar
  • 21
0 votes
1 answer
34 views

Multiple Imputation for Predictors Only, Excluding Missing Outcome Data

I am working with a dataset containing ~300 predictors and ~3000 observations and building a predictive model using elastic net (and hoping to generalize to an external validation set). While the ...
user avatar
  • 5
3 votes
1 answer
90 views

Bayesian priors associated with regularization penalties

I gather that adding a penalty term to (linear) least squares minimization typically corresponds with choosing some prior for Bayes estimation in the normal linear regression model. A couple questions ...
user avatar
1 vote
0 answers
44 views

how do I perform permutation testinging for a prediction model developed within caret package (R)?

I'm fairly new to data science/StackExchange, so please excuse any faux pas I'm trying to perform permutation testing for a chosen ML algorithm (an elastic-net logistic regression) to derive a p-value....
user avatar
  • 21
0 votes
0 answers
37 views

Combining bootstrapping and cross validation for predicting sensitivity to drugs in Barretina et al., 2012

Also posted on https://discourse.datamethods.org/t/combining-bootstrapping-and-cross-validation-for-predicting-sensitivity-to-drugs-in-barretina-et-al-2012/5093. Was curious for any additional inputs/...
user avatar
0 votes
0 answers
20 views

Choosing single lambda and alpha for prediction on independent dataset (Elastic Net)

I have two separate datasets, one of which I am using for training and testing, and the other I am keeping as an independent dataset. My goal is to ultimately train an optimal regression model with ...
user avatar
  • 185
1 vote
1 answer
110 views

Is there a way to write Elastic Net in expanded matrix form?

I am working through a regression problem for a matrix of data that isn't full rank and has more features than observations. For these reasons, I'd like to use elastic net because of its $L1$ and $L2$ ...
user avatar
0 votes
1 answer
39 views

How do I find the optimal values for $\beta$ and $\beta_0$ for sparse linear regression model? Where does the mean of $\lambda$ come into account? [closed]

If someone could point me in the right direction that would be greatly appreciated! Consider the sparse linear regression model: $\min_{\beta_{0},\beta} \left \{ \frac{1}{2}\left \| \beta _{0}e + X\...
user avatar
  • 3
1 vote
2 answers
216 views

Why are my elastic net and lasso r-squared measures negative?

I'm using sklearn.linear_model.Lasso and sklearn.linear_model.ElasticNet on a model that includes a constant. I don't expect a model with a constant to perform worse than the average of the data, ie ...
user avatar
6 votes
2 answers
446 views

Math behind applying elastic net penalties to logistic regression

I understand how Ridge / Lasso / Elastic Net regression penalties are applied to the linear regression's cost function, but I am trying to figure out how they are applied to Logistic Regression's ...
user avatar
  • 63
1 vote
0 answers
24 views

Solving coefficient sum constrained elastic net with quadratic objective term

I am looking for an algorithm to solve an equality constrained elastic net. There are two adaptations I need to make to the standard elastic net. First the objective function includes a quadratic ...
user avatar
  • 11
0 votes
0 answers
18 views

Resources for lasso regression and elastic net

I am interested in learning more about lasso regression and elastic net, can someone provide links that they felt helpful? If there are books that have a step-to-step guide on how to conduct these ...
user avatar
  • 1
2 votes
0 answers
50 views

Minimizing an elastic net loss function

Reading through Andrew Ng's cs229 notes he shows here on page 10 how you can minimize a loss function $J\left( \theta \right)$ representing the sum of least squares by taking the gradient with respect ...
user avatar
  • 21
0 votes
0 answers
79 views

lambda scaling in elastic net regression with glmnet vs sklearn

I am trying to get results to agree between glmnet and sklearn elastic net regression for a specific case where I can't normalise the response variable y. I know that for ridge regression (alpha = 0) ...
user avatar
  • 101
0 votes
0 answers
32 views

Shrinkage regression (Elastic net) produces worse fit on same data when providing more variables compared to previous fit?

I am currently fitting elastic net models on 45 data points (I know, it's not much) in the context of time series analysis. For context, I want to build a forecast model that forecasts 3 months into ...
user avatar
  • 1
3 votes
1 answer
88 views

How can I go from Elastic Net Loss to Scikit-Learn Elastic Net?

I couldn't find a better title, but here's the thing... I was studying Elastic Net regularization and I found this function: $$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \...
user avatar
  • 177
0 votes
0 answers
11 views

elastic net regression excluding few of the factor levels

am very much new to data science and still trying to understand elastic net regression. I have tried using elastic net in R using the code below. my data set is simple and it is for predicting prices ...
user avatar
0 votes
0 answers
90 views

Bivariate analyses vs. Boruta/random forest for removing irrelevant variables prior to penalized regression

I will be using a penalized logistic regression (elastic net) to select variables and their relative importance for predicting an outcome. The goal is to eventually create a risk prediction model, not ...
user avatar
  • 335
2 votes
0 answers
21 views

How/whether to tune Elastic Net parameters using concentration of risk?

Typically, I see alpha and lambda tuned in elastic net models to minimize cross-validated error. Yet, I have seen a handful of articles by one set of authors where they instead tuned parameters to ...
user avatar
  • 335
0 votes
0 answers
105 views

Interpretation of Elastic net having too low or high value of alpha

Often I found the situation that the elastic model what I fitted has optimal alpha value at 0 or 1. Or not only that situation, but also there some alphas go near to 0 or 1.(ex. 0.1 or 0.9) My ...
user avatar
1 vote
1 answer
99 views

How to find out hazard ratio and confidence interval from LASSO cox regression ans plot momogram in R?

I am working on a prognostic model based on time to event endpoint. The training data consists of 800 participants and test data around 400. The number of variables is 21. I was using glmnet package. ...
user avatar
1 vote
0 answers
40 views

Which elastic net to use in Python [closed]

I need to apply the findings of the Zhou, Hastings Paper in Python. Therefore I wanted to use the sklearn elasticnetCV. But is it the same as the elastic net estimate or the naive elastic net estimate?...
user avatar
1 vote
0 answers
23 views

Variables that best discriminate groups based on the glmnet package

I am trying to understand how to interpret the result from the glmnet package. What I ultimately want to find is a set of (influential or important) variables that best discriminates three groups (e.g....
user avatar
0 votes
1 answer
105 views

How to Improve an Elastic Net Model?

I have a dataset with $n=1500$ observations and $p=2700$ variables. I fitted an Elastic Net model with $\alpha=0.4$ and $\lambda=0.1$ I chose the $\lambda$ with cross validation, and the $\alpha$ ...
user avatar
  • 35
0 votes
0 answers
96 views

When to use ridge regression or lasso rather than elastic net?

If one has no ex-ante information about what the L1-ratio hyperparameter should be in the context of elastic net regularization, when should one instead use lasso or ridge? This ratio is referred to ...
user avatar
0 votes
1 answer
159 views

Naive-elastic net and elastic net variable selection comparison

The elastic net paper (here) introduced the naive-elastic net and elastic net. The coefficient estimates of naive-elastic net are obtained by solving the problem $$\hat\beta_{naive-enet}=\text{argmin}...
user avatar
  • 263
0 votes
0 answers
22 views

Hyperparameter estimation on gaussian regression using elastic net

I have been tasked with explaining how a set of 50 environmental variables can affect 300 plus measures of anatomical regions. I am thinking of using gaussian regression as it can handle multiple ...
user avatar
1 vote
2 answers
385 views

Including Fixed Effects in a LASSO/Elastic Net regression model (in R)

So this is a question has vaguely been asked before (see 1 and 2) but I have not been able to find a conclusive answer for anywhere. Essentially I have panel data for 300 US firms between 2012-2020 ...
user avatar
  • 11
0 votes
1 answer
88 views

Is there any example of 2-step predictive model?(i.e. classification model coupled with regression models for each subclass)

I have a large dataset with 10,000+ individuals and many many biological features (>5000). And I want to use these features to build a linear model (e.g. elastic net) to predict their clinical ...
user avatar
0 votes
1 answer
58 views

Can I run glm after MI with Elastic-Net non-zeroed coefficients from 'miselect'?

I have data with n = 80 and 10 predictors, and ran MI using MICE, followed by Variable Selection for Multiply Imputed Data using ‘miselect’ and finally have 4 non-zeroed coefficients. Since ...
user avatar
1 vote
0 answers
29 views

Would a log transformation on my features change the elastic net result

We built the elastic net model on a set of my features and control features. With that, we did various experiments to discuss the importance of the selected features. For example, we showed more of my ...
user avatar
3 votes
1 answer
327 views

How is the standard error calculated for the `lambda.1se` output in the cv.glmnet function?

I understand that lambda.1se is the largest lambda that gives MSE within one standard error of the minimum MSE. But how is the standard error calculated exactly.
user avatar
0 votes
1 answer
71 views

All p-values rendered non-significant in Cox model after initializing coefficients from glmnet

I have a set of 200 genes that are split into numerical high and low, encoded as (1/2). I have set this variable this way for linearity of the model. Also, stratified by cancer and normal cases. I ...
user avatar
  • 192
0 votes
0 answers
395 views

Elastic net regression: Should I tune for lambda at the same time as alpha with cv.glmnet?

I am looking for the best $\alpha$ (=ratio between L1 and L2 penalty) and $\lambda$ (=penalty strength) for my elastic net regression model, using the R package ...
user avatar
  • 1
0 votes
0 answers
126 views

Elastic Net / Lasso as a solution to multiple comparisons and p-hacking? Inferential/Descriptive stats

I have a very large dataset, and I'm trying to find which variable(s) may describe the truth about a certain variable. I've considered just doing OLS on variables that make logical sense. But I've ...
user avatar
  • 105
0 votes
2 answers
32 views

Multicollinearity and case:predictor ratio problems

I have 444 cases and 60ish predictors that suffer from collinearity. The predictors fall into three categories (vol, thickness and demographics). I would prefer to subdivide my cases into 4 (age) ...
user avatar
0 votes
1 answer
47 views

What is the effect of PCA on the Error?

I am fitting an ElasticNet model using an array of values for alpha and l1_ratio. I then plot the result of the negative root mean squared error from cross validation in a heatmap, which gives me the ...
user avatar
  • 3
1 vote
0 answers
228 views

How should I interpret the case where elasticnet gives zero features

I am using elasticnet for the purpose of determining feature importance. In case it is relevant, this is a high-dimensionality model with $n\ll p$. I have seen cases before where Lasso, i.e. logistic ...
user avatar
0 votes
1 answer
78 views

Is the use of elastic net for variable selection purposes a form of data dredging?

Is the use of elastic net for variable selection purposes a form of data dredging? I switched from using stepwise regression procedure for variable selection to elastic net, but I actually do not know ...
user avatar
  • 153
0 votes
1 answer
166 views

The cumulative probability model with ordinalNet, using an elastic net penalty: more coefficients than expected equal to zero

I am trying to fit a cumulative probability model (ordinal logistic regression with 17 categories and 827 observations) with elastic net penalty using the ordinalNet...
user avatar
  • 153
0 votes
1 answer
181 views

What to report in cross validated elastic net regression?

Let's assume I want to construct a regression model to predict a specific outcome variable but I don't have enough data to do a proper train-test set split (n = 200). I have 7 predictor variables (...
user avatar
0 votes
1 answer
316 views

Counterintuitive coefficients in elastic net logistic regression

In a model run of elastic net logistic regression, I encountered a very counterintuitive coefficient. I have looked into the data, model and script, but, I still cannot seem to wrap my head around the ...
user avatar
  • 332
2 votes
1 answer
672 views

What really is 'glmnet' when used in caret in R for binary classification?

like lasso and ridge, elastic net can also be used for classification by using the deviance instead of the residual sum of squares. This essentially happens automatically in caret if the response ...
user avatar
  • 31
2 votes
0 answers
41 views

Strong rules for the elastic net

In their paper (here), Tibshirani et al defined the lasso as the solution to $$ \text{argmin}_{\boldsymbol{\beta}}\frac{1}{2}\left\Vert \mathbf{y}-\mathbf{X}\boldsymbol{\beta}\right\Vert ^{2}+\lambda\...
user avatar
  • 263
2 votes
1 answer
87 views

Is it correct to evaluate individual drivers with the AUC value?

I have a discussion with my supervisor about the usage of AUC to determine, basically, the importance of three different drivers consisting of multiple variables each. He claims I can look into the ...
user avatar
  • 332
0 votes
1 answer
32 views

Correlated predictors with different signs in elastic net

I am struggling with the interpretation of my elastic net results and hope someone might be able to help ... I've done an elastic net regression in R (based on glmnet), with different levels of alpha ...
user avatar
0 votes
1 answer
1k views

How do I compare cv.glmnet models with AIC?

I am using the glmnet package in R, and not(!) the caret package for my binary ElasticNet regression. I have come to the point ...
user avatar
  • 332

1
2 3 4 5 6