# In caret what is the real difference between cv and repeatedcv?

This is similar to question Caret re-sampling methods, although that really never answered this part of the question in an agreed upon way.

caret's train function offers cv and repeatedcv. What is the difference in say doing:

MyTrainControl=trainControl(
method = "cv",
number=5,
repeats=5
)


vs

MyTrainControl=trainControl(
method = "repeatedcv",
number=5,
repeats=5
)


I understand cv breaks the set into k-folds (parameter number), and then starts over and runs it parameters repeats number of times.

The only thing I could think of is that maybe regular cv with repeats uses the same exact indexes for the folds each time? essentially running the cv on the same exact folds each time, vs perhaps repeatedcv selects new folds each times?

Can someone clarify?

• I wonder there are some more methods as well.. need some basic level understanding of each, is there anywhere I could find that? thanks. Jun 19 '18 at 14:04
• In create multi fold, code iterates over multiple times (given by repeats in train Control() syntax in R) for each of the k cross fold (given by number). In cross fold, while using CV, it is a one time process on each of the fold (set by using numbers in train control()). Apr 2 '19 at 5:42

According to the caret manual(see "reference manual"), the parameter repeats only applies when the method is set to repeatedcv, so no repetition is performed when the method is set to cv. So the difference between both methods is indeed that repeatedcv repeats and cv does not.

Aside: Repeating a crossvalidation with exactly the same splitting will yield exactly the same result for every repetition (assuming that the model is trained in a deterministic manner), which is not only inefficient, but also dangerous when it comes to comparing the validation results for different model algorithms in a statistical manner. So be aware of this if you ever have to program a validation yourself.

The actual code behind these parameters can be found in the selectByFilter.R and createDataPartition.R (formerly createFolds.R) source files in the caret/R/' folder of the package.

See these files for e.g. here and here (beware these permalinks may eventually point to older version of the code). For convenience the relevant snippets (as of version 6.0-78 c. Nov 2017) are show below

In selectByFilter.R c. line 157

sbf <- function (x, ...) UseMethod("sbf")
...

"sbf.default" <-
function(x, y,
sbfControl = sbfControl(), ...)
{
...

if(is.null(sbfControl$index)) sbfControl$index <- switch(
tolower(sbfControl$method), cv = createFolds(y, sbfControl$number, returnTrain = TRUE),
repeatedcv = createMultiFolds(y, sbfControl$number, sbfControl$repeats),
loocv = createFolds(y, length(y), returnTrain = TRUE),
boot =, boot632 = createResample(y, sbfControl$number), test = createDataPartition(y, 1, sbfControl$p),
lgocv = createDataPartition(y, sbfControl$number, sbfControl$p))
...


In createDataPartition.R c. line 227

createMultiFolds <- function(y, k = 10, times = 5) {
if(class(y)[1] == "Surv") y <- y[,"time"]
prettyNums <- paste("Rep", gsub(" ", "0", format(1:times)), sep = "")
for(i in 1:times) {
tmp <- createFolds(y, k = k, list = TRUE, returnTrain = TRUE)
names(tmp) <- paste("Fold",
gsub(" ", "0", format(seq(along = tmp))),
".",
prettyNums[i],
sep = "")
out <- if(i == 1) tmp else c(out, tmp)

}
out
}
`
• take a look at their functions....github.com/tonglu/caret/blob/master/pkg/caret/R/… Sep 10 '14 at 23:19
• Could you please give more context in your answer? Links are good, but we try to avoid answers that won't stand on their own -- links can disappear. Sep 11 '14 at 0:54

Admittedly, this is a VERY old post, but based on the code snippets provided by user3466398, the difference is that repeatedcv does exactly that: it repeatedly performs X-fold cross-validation on the training data, i.e. if you specify 5 repeats of 10-fold cross-validation, it will perform 10-fold cross-validation on the training data 5 times, using a different set of folds for each cross-validation.

The rationale for doing this, I presume, is to allow one to have a more accurate and robust accuracy of the cross-validation testing, i.e. one can report the average CV accuracy.