Repeated CrossValidation, finalModel and ROC curves I got a problem understanding the meaning of the finalModel when using a repeated CV. 
ctrl = trainControl(method="repeatedcv", number=10, repeats = 300, savePredictions = TRUE, classProbs = TRUE)
mdl = train("Label~.", data=Data, method = "glm", trControl = ctrl)
pred = predict(mdl, newdata = Data, type="prob")
roc.1 = roc(Data$Label, pred$control)
roc.2 = roc(mdl$pred$obs,mdl$pred$control)

from what I understand :


*

*In roc.1 I tested the "finalModel" of the rCV training on all my database then build the ROC curve associated to this "finalModel".

*In roc.2 I build the "average" ROC curve using all the rCV process results.


What I don't get is: what does the "finalModel" represent? 


*

*Is it a model averaging all the trained models coefficients?

*Is it correct to use it a predictive model (on both the training dataset and a different set)?

 A: For all caret models, the final model is trained on the full dataset.  caret::train uses the cross-validation scheme you chose to select model parameters (e.g. mtry for a random forest) and estimate out-of-sample performance of the model.  Once the cross-validation is done, caret retrains the model on the full dataset, using the parameters it selected during cross-validation.
So roc.1 is an in-sample roc curve.
The model does not average the trained model's coefficients.  It re-fits the model on the full dataset.
It is NOT correct to use the final model on the training data, but it is correct to use on a different dataset.
A: So finally to summarize :
ctrl = trainControl(method="repeatedcv", number=10, repeats = 300, savePredictions = TRUE, classProbs = TRUE)
mdl = train("Label~.", data=Data, method = "glm", trControl = ctrl)
pred = predict(mdl, newdata = Data, type="prob")
roc.1 = roc(Data$Label, pred$control)
roc.2 = roc(mdl$pred$obs,mdl$pred$control)
roc.3 = roc(as.numeric(mdl$trainingData$.outcome=='case'),aggregate(case~rowIndex,mdl$pred,mean)[,'case'])



*

*roc.1 is irrelevant as it evaluates a model on the same data used to train it (the finalModel is just the fit on Data ignoring the CV argument, built to apply on a different dataset for future prediction)

*roc.2 is 'almost' accurate as it will consider each prediction independently (averaging the prediction, not the probabilities)

*roc.3 is the correct way to do it as it averages the prediction probabilities for each sample among the repeated CV (contrary to roc.2 where the prediction results are averaged)

