1
$\begingroup$

I have a question about cross-validation and how the beta values are returned by cvglmnet.

First, I understand that when using 10-fold cross validaiton for optimal parameter search, you take the parameters from the fold that gave the highest performance (in my case AUC). And this might be done preferrably after using nested cross validation in advance to get an overall estimate of model performance/stability.

Second, I understand that when you use cvglmnet, it automatically does cross validation and returns the beta values at each grid point of attempted lambda values. Lambda.min is chosen based on which lambda value gave the higest cvm (the mean cross-validated AUC?). So, in this case, the lambda value that results in the higest average performance across all 10 folds is chosen?

Then what exactly are the beta values for each lambda value in the 'beta' matrix (returned by cvglment). Is it:
(a) the beta coefficients of the one fold that gave the highest perfomrnace at that lambda value, or
(b) the average of the beta coefficients across all 10 folds at that lambda value?

I believe that according to my understanding of how optimal parameter selection in cross-validation works, it should be (a), but so far it seems that cvglmnet seems to be doing (b).

$\endgroup$
1
$\begingroup$

Update: I looked again at the documnetataion for glmnet and found that the beta returned by cvglmnet (CVerr.glmnet_fit) is "a fitted glmnet object for the full data".

So I think this answers my question.. The beta coefficients returned by cvglmnet is neither (a) nor (b), but it is the result of running glmnet again on the whole training dataset.

I think I had also misunderstood how to select optimal parameters from cross-validation as I mentioned in my second paragraph. We do not use the optimal parameters (lambda, alpha, beta-coefficients) from just one fold that gave the highest performane, but rather retrain the whole training dataset with the hyperparameters (lambda, alpha) to get a new set of beta-coefficients. And this is the beta returned by cvglmnet?

If anyone could confirm that what I found is correct, that would be very appreciated!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.