3
$\begingroup$

So for cv.glmnet we get different values of lambda (lambda.min and lambda.1se) due to the randomness in how the data is split.

Is it reasonable to repeat cv.glmnet many times and take the mean value of Lambda. Or would it be better to run cv.glmnet with LOOCV instead of 10-fold CV.

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes it is reasonable. I don't think lambda.1se adds anything.

For any cross validation metric it is recommended to run multiple cross validation runs.

Similarly how many folds is a generic problem.

However I would point out your reasoning is wrong: loo has high variance and low bias, basically because the estimators are so correlated, so noise doesn't cancel out. Low bias because we are estimating test error for model with n data points with model with n-1.

Page 242 of elements of statistical learning (available online)

What value should we choose for K? With K = N, the cross-validation estimator is approximately unbiased for the true (expected) prediction error, but can have high variance because the N “training sets”are so similar to one another. The computational burden is also considerable, requiring N applications of the learning method. In certain special problems, this computation can be done quickly

$\endgroup$
2
  • $\begingroup$ Ok thanks, and what do you mean by lambda.1se doesn't add anything. $\endgroup$
    – Dylan Dijk
    Feb 26, 2021 at 8:33
  • $\begingroup$ that there is nothing specific about lambda.1se, the question is generic - how to estimate a metric using crossvalidation $\endgroup$
    – seanv507
    Feb 26, 2021 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.