I am sorry if there is an obvious or intuitive answer to this, which I missed.
We have tuned the hyperparameters of a RF using Grouped 10 - Fold CV (repeated 5 times), to obtain the values for mtry and ntree which result in the highest accuracy.
I understand the concept of (grouped) K-Fold Cross Validation. However my first basic question is how is the hyperparameter tuning from caret incorporated into this resampling technique? In a toy example where we want to test
mtry = c(1,2) and
ntree = c(500,1000) (meaning 4 possible combinations of hyperparameters) would caret run the complete 10 Fold CV repeated 5 times, for each of the 4 possible combination of mtry and ntree?
Or does caret iterate through the different hyperparameter combinations in the repeats of the CV?
Furthermore if we want to make predictions about a novel dataset (df2) based on the rf model (output from caret hyperparam tuning), which model will be used? Since in a 10-Fold CV there are 10 models trained on (10-1 Fold of data) and evaluated in 1 fold. I know that the performance metrics are averaged, however I can't really imagine that the same is true for the model.
transfer <- predict(rf.df1,newdata = df2) confusionMatrix(df2$Class,transfer) #which model is used here for the predictions?