0
$\begingroup$

I am confused about how exactly cvglmnet calculates the "optimal" hyperparameters (specifically, lambda.min and lambda.1se).

As far as I understand, k-fold cross validation in itself doesn't give us an "optimal hyperparameter", but rather an estimate of the model's overall performance on other datasets. And cross-validation is what the documentation says that cvglmnet does.

However, isn't cvglmnet providing us with a lambda.min and lambda.1se a form of hyperparameter search? So would the results of running cvglmnet effectively be similar to nested cross validation?

Any thoughts are appreciated!

Thanks, Michelle

|cite|improve this question
$\endgroup$
  • $\begingroup$ Although this question is obviously R-related, I think it's probably on topic here because it relates to the underlying operations done by cv.glmnet and not say, how to call it or structure its inputs. $\endgroup$ – Matt Krause Feb 14 '17 at 21:17
1
$\begingroup$

cross validation is a tool, which you can use for either parameter selection or test prediction error. If you want both you need to do nested cross validation ( since if you use cross validation to choose parameters, then the 'best' test error has been selected).

you can see the code for cv.glmnet either on eg github or by typing cv.glmnet ( whilst the heavy duty functions are written in ?fortran, a lot of the wrapper functions eg crossvalidation is done in R)

so cv.glmnet does crossvalidation for parameter selection. You will have to 'roll your own' nested crossvalidation to also get an estimate of the test error

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for your input. So then if I wanted to find an estimate of the test error of the model with best parameters, I would do nested CV on the training set (with cvglment nested inside for parameter selection)? Next if I want to apply those parameters from the model to another test dataset and see how well it performs there, then would I used only cvglmnet on the whole training set and take the best parameters from there and apply them to the test dataset? $\endgroup$ – Michelle Mar 23 '17 at 2:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.