2
$\begingroup$

I am using elastic net regression to build explanatory models of clinical outcome measures using mutually correlated predictors. In elastic net regression, does choosing the regularisation parameter (Lambda) by cross-validation, in order to minimise MSE, prevent overfitting? Do the degrees of freedom provide an indication of overfitting when N is small?

$\endgroup$

1 Answer 1

2
$\begingroup$

More often it reduces over-fitting by shrinking the coefficient estimates. But the lowest cross-validated MSE estimate over the regularization parameter is optimistic when taken as an estimate of the out-of-sample performance of the model using that value of the regularization parameter; though not usually by much, as the optimization is very constrained. Use cross-validation or bootstrap validation of the whole procedure to estimate how much.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks for the answer! I was using the models to try and explain the outcomes measures using the predictors and so was trying to avoid out of sample estimates (different model) each time. Is the optimistic bias increased with p>=N? $\endgroup$
    – BGreene
    Commented Feb 11, 2015 at 18:13
  • $\begingroup$ See here. $\endgroup$ Commented Feb 13, 2015 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.