1
$\begingroup$

Does predicting on a test set with an ordinary linear regression model result in a smaller predictive MSE compared to a penalized regression model (LASSO or ridge)?

$\endgroup$
3
  • $\begingroup$ if that were the case, why would we have Lasso or Ridge? $\endgroup$ – user75138 Oct 21 '15 at 1:56
  • $\begingroup$ I was under the impression that using ridge reduces the variance of the estimates $\endgroup$ – K23 Oct 21 '15 at 2:02
  • $\begingroup$ That is correct, but that will also increase the bias. The goal is to minimize the MSE by adding a little bias and reducing the variability substantially. $\endgroup$ – user75138 Oct 21 '15 at 2:37
1
$\begingroup$

OLS can be considered a special case of penalized regression where the penalty strength is zero. In this context, we choose the penalty strength in an attempt to minimize generalization (predictive) error, using cross validation. In this sense, we generally expect penalized regression to have superior predictive strength.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.