Ridge regression shrinks regression coefficients by a set proportion, and never to zero. Given this, its key benefit seems to be to reduce prediction error by decreasing the sampling variability of the parameter vector, at the cost of some bias.

Given data sets of 100,000 records or larger, the sampling variability of the parameter error will tend to zero. In light of this, is there any benefit in actually performing ridge regression on such big data sets?

  • $\begingroup$ I guess it would depend on the signal-to-noise ratio, but with ever larger data sets the benefits of shrinkage should shrink (pun intended). $\endgroup$ – Richard Hardy Jan 26 '17 at 15:09
  • $\begingroup$ It would also depend on other details, such as if there are error in measurement of the predictors. $\endgroup$ – kjetil b halvorsen Jan 26 '17 at 16:16
  • $\begingroup$ A) it depends on the complexity of your model, non linear transformation s etc. B) at least in e-commerce, big data is skewed. Eg 90% of song plays come from 100 artists ( made up numbers). $\endgroup$ – seanv507 Jun 15 at 8:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.