I am using elastic net regression to build explanatory models of clinical outcome measures using mutually correlated predictors. In elastic net regression, does choosing the regularisation parameter (Lambda) by cross-validation, in order to minimise MSE, prevent overfitting? Do the degrees of freedom provide an indication of overfitting when N is small?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
More often it reduces over-fitting by shrinking the coefficient estimates. But the lowest cross-validated MSE estimate over the regularization parameter is optimistic when taken as an estimate of the out-of-sample performance of the model using that value of the regularization parameter; though not usually by much, as the optimization is very constrained. Use cross-validation or bootstrap validation of the whole procedure to estimate how much.
answered Feb 6, 2015 at 21:19
Scortchi - Reinstate Monica♦Scortchi - Reinstate Monica
29.1k8 gold badges90 silver badges273 bronze badges
-
1$\begingroup$ Thanks for the answer! I was using the models to try and explain the outcomes measures using the predictors and so was trying to avoid out of sample estimates (different model) each time. Is the optimistic bias increased with p>=N? $\endgroup$– BGreeneFeb 11, 2015 at 18:13
-