# Questions tagged [ridge-regression]

A regularization method for regression models that shrinks coefficients towards zero.

699 questions
Filter by
Sorted by
Tagged with
4 views

### Interpreting Shapley Values on Breast Cancer

I was analyzing Shapley Values on the Wisconsin breast cancer data set (binary classification). I applied it on Random Forest and on Ridge and Lasso Regression. However the summary plot seems to be ...
• 1
60 views

### Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
17 views

• 247
265 views

### Ridge Trace Plot - Interpretation

In my research, I aimed to perform a regression model with four predictors and one response variable. When I verified a high collinearity among the predictors, I was instructed to handle this problem ...
• 47
1 vote
114 views

### Feature Selection via RFE with Ridge or SVM (Regression)

I have a regression problem where the number of samples n is less than the number of features p (e.g., p=500 and n=400; but the problem can be extended to p=3000+ and n=400). The features are largely ...
• 13
60 views

### Ridge regression with shrinkage towards nonzero matrix

Suppose I want to perform ridge-regularized linear regression, except that we shrink the coefficients to a nonzero matrix: $$W^* = \arg\min_W \|Y - X W \|^2_2 + \lambda\|W-W_0\|^2_2.$$ However, I ...
92 views

### How to obtain odds ratio (and 95% CI) from ridge regression model

I am currently working on a ridge logistic (predictive) model. I was able to complete most of the steps and obtain the coefficient but I keep getting an error message when it comes to the odds ratio &...
• 311
58 views

• 127
102 views

### Is the modeling strategy of GAM in MGCV equivalent to ridge regression when there are no smoothing terms?

According to GAM, it utilizes a penalized likelihood, which is maximized by penalized iteratively re-weighted least squares (P-IRLS), to obtain parameter estimations. The likelihood is defined as: ...
• 31
50 views

### thresholding prior to model evaluation

Methodology question. The ML textbook approach is this: perform model fit - optimisation assess fit with Cross-Validation tune decision rule by thresholding on the prediction probability (...
195 views

### Variance of the ridge regression estimator

I have some concerns about the image below (note that $\mathbf W_{\lambda} = (\mathbf X^\top \mathbf X + \lambda \mathbf I)^{-1} \mathbf X^\top \mathbf X$): My main concern is that this derivation of ...
• 531
1 vote
89 views

### Expected value of the ridge regression estimator

I am trying to understand this derivation: I think everything except the last equality is fairly simple, but I do not understand the last equality. Is there an error here? I appreciate any help.
• 531
83 views

### Lasso and SGDRegressor are not working well

I want to fit some data using Lasso, Ridge and SGDRegressor and to compare the results. ...
37 views

### Bayesian Elastic Lasso

While studying elastic lasso, I have had a thought if I can apply a Bayesian method to the Elastic Lasso. If I want apply Bsyesian way of making a Regression model with Elastic Lasso, what do I need ...
• 2,585
7 views

### Appropriate regression framework for evaluating best players

I would like to model the best performers in a game using a ridge regression approach similar to RAPM in the NBA. Background: The game involves two teams (say team A and team B) of five players each. ...
32 views

### Why does the ridge penalty shrink the singular values? [duplicate]

I am trying to understand the following analysis of ridge regression. I am new to SVD but I think I have a sufficient grasp on most of the content. There are two things I am struggling with. The ...
• 531
36 views

### In Ridge/Lasso Regression, What's The Advantage To Using CV Lamda And Then Some Form Of Training/Testing

When running a lasso or ridge regression, cross-validation allows us to find an optimal (minimized lamda.) So - if we were using glmnet with a logistic response variable... ...
65 views

### Is Individual Coefficient Significance with Ridge or Lasso possible, when Amount of Variables exceeds Observations

First, to introduce you to my situation, I have a dataset containing n = 16 observations and p = 17 variables. My variable set contains 16 independent variables (14 variables I'm interested in and two ...
1 vote
45 views

### Sign change in LASSO and RIDGE of coefficients

I am estimating in total three models: Logistic regression without any penalization (as benchmark model), logistic regression with L1 penalization (LASSO) and with L2 penalization (RIDGE). Now i ...
• 27
33 views

### Is it possible (reasonable) to weight the regularization for some variable in Ridge/elastic net based on their importance/causal effect

Say I have 100 predictor variables. And I have estimations from a causal inference method that indicates the causal effect size of each variable to the response variable. Then I want to build a linear ...
35 views

### Proof of invariant angle between $Y$ and $\hat Y$ in $L^2$ regularisation

On this site is the following question which claims that the $L^2$ regularised OLS preserves the angle between $\hat Y$ and $Y$ irrespective of the value $\lambda$. I have not found any source that ...
• 315
319 views

### Lasso vs Ridge Regression

My question relates on the Ridge vs Lasso Regression. I know the difference in the cost function (ridge penalizes sum of quadratic coefficients, lasso penalizes sum of absolute value of coefficients). ...
• 27
41 views

### Minimizing $L_2$ norm with constrained residual sum of squares (RSS)

I have some complex-valued time-series data, $y \in \mathbb{C}^n$ - a signal with additive Gaussian white noise. The goal is to find the Fourier coefficients of this signal. Ideally, you would just do ...
• 163
I am trying to derive the gradient and hessian of logistic regression with ridge penalty. The log-likelihood should be (correct me if I am wrong): \sum_{i=0}^n\Big(\log{(P_i^{y_i}(1-P_i)^{1-y_i}- \...