0
$\begingroup$

Generally, 10-fold CV is used to find the best $\lambda$ (the shrinkage parameter, not the trade-off between L1 and L2 norm $\alpha$ - see https://stats.stackexchange.com/a/64278/185237) in an elastic net

After CV, we then chose the Lambda value that yields results within one StandardError (SE) of the MeanSquaredError (we prefer 1SE over MSE, because it has shown to retain more generality) I am currently using Matlab, and it has this functionality already included, see documentation here: https://uk.mathworks.com/help/stats/lasso.html

So far so good, but here's my question: Across folds, the specific $\lambda$ value can vary - so how do I compare them in CV to calculate a $\lambda$ value's MSE/1SE value? If I have 5 $\lambda$ values per fold, I need to be able to calculate each value's mean across each fold, so that I can pick the best one in the end to use, right? But for example the first $\lambda$ value in Fold 1, can vary from the first $\lambda$ value in Fold 2.

If I understood the original paper correctly (https://www.cc.gatech.edu/people/home/isbell/classes/reading/papers/elasticnet.pdf), then $\lambda$ can be found via a step function (instead of varying the value itself, we increase the k which in turn yields a $\lambda$ value) - so does this mean I can compare the error for the highest $\lambda$ value in Fold 1 with the highest $\lambda$ value in Fold 2, even though they are different $\lambda$ values, because they represent the same step?

My motivation is the following: I would like to re implement this CV procedure, so that I can carry out some other calculations with each fold. Specifically, I would like to apply Mutual Information Feature Selection on this same fold, to be able to compare within fold, which features are selected by the two different methods.

Grateful for any input or help where I can read up on it - thank you!

$\endgroup$
  • $\begingroup$ Could you please clarify the nature of the CV that you wish to perform? It's not clear to me from your question whether you are talking about a single 10-fold CV, from which you wish to gather information from within each fold, or some form of multiple or nested CV. $\endgroup$ – EdM Mar 28 at 16:45
  • $\begingroup$ Hi - the CV is used to determine the a good lambda parameter It's a single 10 fold, where afterwards the lambda value that had the best MSE/1SE value can be chosen - see my answer below for the answer to my question $\endgroup$ – CrankMuffler Mar 28 at 17:52
0
$\begingroup$

I figured it out - the lambdas are generated once for the whole dataset, and then the same values are used in all folds of the CV procedure

This ensures that each fold uses the same values for their calculations

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.