Linked Questions

1
vote
1answer
1k views

Non-Singularity due to inclusion of non-zero lambda in ridge regression [duplicate]

There were many similar questions on this site , related to this but none were exactly to the point I wanted to ask So the question is relates to ridge regression and This link where there is a ...
2
votes
1answer
269 views

Why increasing lambda parameter in L2-regularization makes the co-efficient values converge to zero [duplicate]

Why increasing lambda parameter in L2-regularization makes the co-efficient values converge to zero? I have just tried to do the math, but it's a little bit rusted. Lets say that we have a simple ...
0
votes
1answer
34 views

Adding more samples to ordinary regression is equall to ridge regression [duplicate]

I am a beginner in machine learning. I have a question why adding more samples to a data set is equal to adding regularization term to the ordinary least squares loss function? (In other words why can ...
7
votes
2answers
7k views

Ridge regression in R with p values and goodness of fit

Doing ridge regression in R I have discovered linearRidge in the ridge package - which fits a model, reports coefficients and p ...
4
votes
3answers
5k views

Ridge Regression -Increase in $\lambda$ leads to a decrease in flexibilty

In Introduction to Statistical Learning, in the part where ridge regression is explained, the authors say that As $\lambda$ increases, the flexibility of the ridge regression fit decreases, ...
8
votes
3answers
2k views

How to perform non-negative ridge regression?

How to perform non-negative ridge regression? Non-negative lasso is available in scikit-learn, but for ridge, I cannot enforce non-negativity of betas, and indeed, ...
7
votes
2answers
473 views

Lasso penalty only applied to subset of regressors

This question has been asked before but there were no responses, so I thought I might ask again. I'm interested in applying a Lasso penalty to some subset of the regressors, i.e. with objective ...
6
votes
3answers
358 views

Is there any special case where ridge regression can shrink coefficients to zero?

Are there some special cases, where the Ridge Regression can also lead to coefficients that are zero ? It is widely known, that lasso is shrinking coefficients towards or on zero, while the ridge ...
1
vote
3answers
468 views

Why are solution to ridge regression always expressed using matrix notation?

Consider the following ridge regression problem: minimize the loss function $\sum_{i=1}^n ||y_i - w^T x_i||_2^2 + \lambda ||w||_2^2$ with respect to the weight vector w. Taking derivative with respect ...
3
votes
1answer
1k views

double feature value in ridge regression, coefficients change?

In ridge regression using unnormalized features, if you double the value of a given feature A (i.e., a specific column of the feature matrix), what happens to the estimated coefficients for every ...
1
vote
0answers
1k views

Deriving the gradient of a loss function for generalized logistic regression

I am trying, without much success so far, to derive the gradient of the following cost function in order to fit a logistic curve to some data: $J(a, k, b, m) = \sum_i^n(y_i - a + \frac{k - a}{(1 + e^{...
4
votes
1answer
295 views

Is there a “fused” version Ridge regression?

we know there is a fused version of LASSO. Fused LASSO adds a further regularizer demanding the smoothness of \beta. More details could be found here I am wondering why I cannot find something ...
10
votes
1answer
446 views

LASSO relationship between $\lambda$ and $t$

My understanding of LASSO regression is that the regression coefficients are selected to solve the minimisation problem: $$\min_\beta \|y - X \beta\|_2^2 \ \\s.t. \|\beta\|_1 \leq t$$ In practice ...
4
votes
1answer
257 views

Modifying ordinary least squares (OLS) in ridge regression to perform transfer learning

I have a question on using ridge regression for transfer learning. Transfer learning is a type of Machine Learning where knowledge from the source domain when performing a task is transfered to the ...
0
votes
1answer
257 views

Applying the normal equation to a ridge regression proof?

A previous answer to a question asking for a derivation of ridge regression points out at one juncture that from the following equation: $$(y_∗−X_∗β)′(y_∗−X_∗β)=(y−Xβ)′(y−Xβ)+λβ′β$$ It follows that ...

15 30 50 per page