# Questions tagged [ridge-regression]

A regularization method for regression models that shrinks coefficients towards zero.

683 questions
Filter by
Sorted by
Tagged with
24 views

### How to obtain odds ratio (and 95% CI) from ridge regression model

I am currently working on a ridge logistic (predictive) model. I was able to complete most of the steps and obtain the coefficient but I keep getting an error message when it comes to the odds ratio &...
16 views

### Ridge regression - prove derivate is zero at Q* [closed]

How to I prove that the optimal point in L2 ridge regression will make the derivative 0? It's kind of reverse work( I've substituted the q* value in the gradient but not able to prove its 0. L(Q) = 1/...
30 views

28 views

### Is the modeling strategy of GAM in MGCV equivalent to ridge regression when there are no smoothing terms?

According to GAM, it utilizes a penalized likelihood, which is maximized by penalized iteratively re-weighted least squares (P-IRLS), to obtain parameter estimations. The likelihood is defined as: ...
40 views

### thresholding prior to model evaluation

Methodology question. The ML textbook approach is this: perform model fit - optimisation assess fit with Cross-Validation tune decision rule by thresholding on the prediction probability (...
51 views

### Variance of the ridge regression estimator

I have some concerns about the image below (note that $\mathbf W_{\lambda} = (\mathbf X^\top \mathbf X + \lambda \mathbf I)^{-1} \mathbf X^\top \mathbf X$): My main concern is that this derivation of ...
29 views

### Expected value of the ridge regression estimator

I am trying to understand this derivation: I think everything except the last equality is fairly simple, but I do not understand the last equality. Is there an error here? I appreciate any help.
13 views

### Lasso and SGDRegressor are not working well

I want to fit some data using Lasso, Ridge and SGDRegressor and to compare the results. ...
33 views

### Bayesian Elastic Lasso

While studying elastic lasso, I have had a thought if I can apply a Bayesian method to the Elastic Lasso. If I want apply Bsyesian way of making a Regression model with Elastic Lasso, what do I need ...
6 views

### Appropriate regression framework for evaluating best players

I would like to model the best performers in a game using a ridge regression approach similar to RAPM in the NBA. Background: The game involves two teams (say team A and team B) of five players each. ...
27 views

### Why does the ridge penalty shrink the singular values? [duplicate]

I am trying to understand the following analysis of ridge regression. I am new to SVD but I think I have a sufficient grasp on most of the content. There are two things I am struggling with. The ...
20 views

### In Ridge/Lasso Regression, What's The Advantage To Using CV Lamda And Then Some Form Of Training/Testing

When running a lasso or ridge regression, cross-validation allows us to find an optimal (minimized lamda.) So - if we were using glmnet with a logistic response variable... ...
40 views

### Is Individual Coefficient Significance with Ridge or Lasso possible, when Amount of Variables exceeds Observations

First, to introduce you to my situation, I have a dataset containing n = 16 observations and p = 17 variables. My variable set contains 16 independent variables (14 variables I'm interested in and two ...
25 views

### Sign change in LASSO and RIDGE of coefficients

I am estimating in total three models: Logistic regression without any penalization (as benchmark model), logistic regression with L1 penalization (LASSO) and with L2 penalization (RIDGE). Now i ...
19 views

### Is it possible (reasonable) to weight the regularization for some variable in Ridge/elastic net based on their importance/causal effect

Say I have 100 predictor variables. And I have estimations from a causal inference method that indicates the causal effect size of each variable to the response variable. Then I want to build a linear ...
27 views

### Proof of invariant angle between $Y$ and $\hat Y$ in $L^2$ regularisation

On this site is the following question which claims that the $L^2$ regularised OLS preserves the angle between $\hat Y$ and $Y$ irrespective of the value $\lambda$. I have not found any source that ...
140 views

### Lasso vs Ridge Regression

My question relates on the Ridge vs Lasso Regression. I know the difference in the cost function (ridge penalizes sum of quadratic coefficients, lasso penalizes sum of absolute value of coefficients). ...
32 views

### Minimizing $L_2$ norm with constrained residual sum of squares (RSS)

I have some complex-valued time-series data, $y \in \mathbb{C}^n$ - a signal with additive Gaussian white noise. The goal is to find the Fourier coefficients of this signal. Ideally, you would just do ...
33 views

### How exactly does the glmnet in R determine the penalty in ridge regression?

in R, once I call https://www.rdocumentation.org/packages/glmnet/versions/4.1-2/topics/cv.glmnet with alpha = 0, I will magically get a set of coefficients from ridge regression, without having to ...
91 views

197 views

### When there are more variables than observations do shrinkage methods (such as Ridge and Lasso) always find a solution?

Assume we have $n$ observations and $p$ explanatory variables we want to model. To apply ridge regression, we choose a constraint parameter $\lambda \geq 0$ and estimate the coefficients $\beta_i$ ...
87 views

### Is there any justification for not standardizing predictors on disparate scales when using Lasso/Ridge?

I've looked at some Kaggle notebooks lately of people using Lasso/Ridge for linear regression. The majority that I've seen don't seem to standardize the predictors before they fit Lasso/Ridge even ...
69 views

### Why does area under curve not change from 0.5?

I have performed a ridge logistic regression with glmnet and now I look at the performance metric AUC. The script is: ...
45 views

### How to optimise penalty parameter in ridge regression using AIC

So I know for a ridge regression model, we need to find an optimal $\lambda$ value. I also know that we can achieve this by finding an optimal AIC value, that is, we find the $\lambda$ value that ...
83 views

### LassoCV vs RidgeCV in Python -- why are their default number of folds different?

In https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LassoCV.html, it says that LassoCV defaults to 5 folds. In https://scikit-learn.org/stable/...
19 views

### How do the training and cross validation mean squared error curves behave as a function of $\lambda$?

I am currently looking into methods of choosing optimal tuning parameter $\lambda$ for ridge regression. I think that for the cross-validation the MSE should be relatively high for $\lambda=0$. Then I ...
I am trying to figure out how the solution for ridge regression changes when the error term is independent but NOT identically distributed such as $\mathbb\epsilon = \mathcal{N}(0, \Sigma)$ rather ...