# Questions tagged [ridge-regression]

A regularization method for regression models that shrinks coefficients towards zero.

787 questions
Filter by
Sorted by
Tagged with
1 vote
70 views

### Can I utilize Ridge Regression to update coefficients of a Linear Regression model for a new dataset?

I have fitted a Linear Regression Model using one dataset. Now, I have another smaller dataset that I want to refine the model with. Can I use Ridge regression to update the estimated coefficients for ...
• 135
7 views

### Computing Test Loss in Kernel Ridge Regression

In Kernel Ridge regression we have the standard loss function $$L(\beta) = \|Y-K\beta\|_2^2 + \alpha \beta^T K \beta$$ Here, $K$ is the kernel (gram) matrix. If I compute $\beta$ on a training set, so ...
• 1,481
20 views

• 335
28 views

### Effective degrees of freedom for residual variance in ridge regression

The definition of the effective degrees of freedom (dof) in Ridge Regression via the trace of the "hat matrix" is well known (see e.g. Hastie and Tibshirani's Generalized Additive Models). ...
• 1,008
12 views

### The relationship between ridge regularization and CNN Data Augmentation

In Chapter 10.3.4 of Introduction to Statistical Learning with Applications in Python by James et al. there is a sentence on data augmentation for CNNs (adding natural transformations of images into ...
1 vote
43 views

### Why does ridge regression apply a non-monotone transformation to the singular values of the design matrix?

Per Wikipedia, Ridge Regression is equivalent to transforming the singular values $\sigma_i$ of the design matrix to $\frac{\sigma_i^2 + \alpha^2}{\sigma_i}$, where $\alpha$ is (in Wikipedia's ...
• 121
38 views

### How is the weight vector calculated when using kernel trick for ridge regression

Im trying to understand how kernelized ridge regression works, and how we manage to first transform, and subsequently learn on higher-dimensional features without explicitly having to calculate them. ...
• 33
140 views

### What are a priori advantages of Lasso regularization for linear regression models?

What are a priori advantages of Lasso regularization for linear regression models, over many other heuristically-justifiable methods that both regularize the problem and perform variable selection? ...
• 31
94 views

• 3,972
22 views

### Ridge regression gives me this plot. How to interpret it? [duplicate]

I have done this plot with cv.glmnet(), can someone help me to interpret it? I also noticed that I get 2 different lambdas: lambda.min and lambda.1se What is the difference between these lambdas? Why ...
38 views

### Coordinate Descent Alternating between LASSO and Ridge

Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient? For example, apply a LASSO constraint to a predefined 3 variables and Ridge to ...
• 1,552
78 views

### Using Ridge Regression to estimate importance of multicollinear variables in python

New to statistical analysis so bear with me. I have a dataset with 1 (say y) dependent variable and 5 independent variables (say x1,x2,x3,x4,x5) which are highly correlated. I know that y depends (...
• 101
87 views

### What's the effect of doubling the data and copying the data for Lasso Regression and Ridge Regression on standard error?

Suppose we have a dataset $X$, where each piece data of $X$ is a row vector, and the data generation process satisfies Gaussian-Markov assumption. If we do ridge regression on $Y\sim 2X$, how does the ...
1 vote
15 views

### Increasing accuracy of prediction

I'm working with this data set trying to implement a model to predict the variable normexam. I've used the following models on sklearn, adding dummies for categorical variables, and got the following ...
• 133
51 views

• 890
142 views

### Coefficient of highly correlated variables under LASSO and ridge

I have been presented with some interesting questions but unfortunately, I am struggling to provide satisfactory answers. The questions are as follows: How will the regression coefficients of two ...
• 307
303 views

### Interpreting the biased coefficients of a ridge or LASSO regression model [duplicate]

In a recent conversation with one of the colleagues I was presented with a view that LASSO/Ridge regularization (trading bias for variance) renders coefficient estimates useless for interpretation, i....
90 views

### Question about retraining a regression model

Exercise. Suppose you train a Ridge model to a regression problem that has a normalized perfomance measure (say K) that attains a value in the interval [0,1], where 0 means that the model is terrible ...
• 185
I am trying to derive $\widehat{\beta}$ in summation form from the following: $$\text{argmin } \sum_{i=1}^{N}(y_i - X_i^{T}\beta)^2 + \lambda \sum_{k=1}^{K}{\beta}_k^2$$ I do not want to resort to ...