I'm a little confused here. Appreciate if anyone can help me out.
During a k-fold nested CV, I understand for each combination of training fold and testing fold , the training fold will be further split into k subsets and a small CV will be carried out to determine the optimal hyper-parameters.
My question is: are the models generated by each training fold the same? If not, which model should be used when apply to productive environment.
I know there is a similar thread 'Nested cross validation for model selection'. But I still don't quite get it after reading for several times. Please bear with me as I just step into the field.
Thanks in advance.
After reading some other threads, I can understand the mechanism below:
- Create hyper-parameter matrix;
- Use sub-training set to fit model;
- Use fitted model and validation set to select model;
- Use test fold to evaluate the performance of the chosen model.
However, my question is triggered by the book 'Data Science for Business'. When it describes 'Nested Cross Validation', it says:
... before building the model for each fold, we take the training set and first run an experiment: we run another entire cross-validation on just that training set to find the value of C estimated to give the best accuracy. The result of that experiment is used only to set the value of C to build the actual model for that fold of the cross-validation...
In my understanding, each fold may get a different value of C which leads to different models. Then which model should I use for productive operation?
Please help point out anything wrong in above understanding or if I have any misunderstanding of the book. Thanks a lot.