I did linear regression with and without regularization parameter (ridge) and found that regularization improves the regression accuracy for just some of my test data and error goes up for the rest. So it seems that regularization is destructive in some cases. So I am thinking to make ridge parameter as a function of "something" and this gets checked in the code. If it's found that regularization is not needed, the parameter lambda is set to zero otherwise to another predefined value. So I am wondering if there is any way that I can detect in a code if regularization needed or not? How can I check this?

  • 1
    $\begingroup$ Use cross validation to select the ridge penalty. In the case where it's not beneficial, you'll find the optimal penalty is 0, which corresponds to ordinary least squares regression. $\endgroup$ – user20160 Jan 22 '17 at 4:40
  • $\begingroup$ I did the same but the point is that I can't find a value for lambda to make all testing data (2000) better with respect to OLS. Some data becomes much better and some worse. I am trying to figure out why those data gets worse and then add a "if" to my code such that regularization is applied if this condition is met. $\endgroup$ – user3720389 Jan 22 '17 at 4:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.