NxN- fold cross validation

I just stumbled over the expression NxN- fold cross validation and i am not sure what it's exact meaning.

N-fold cross validation, as i understand it, means we partition our data in N random equal sized subsamples. A single subsample is retained as validation for testing and the remaining N-1 subsamples are used for training. The result is the average of all test results.

Now my question is what does NxN-fold cross validation mean?

• Can you share the article or other resource where you found the expression $N \times N$-fold cross-validation used?
– Sycorax
Nov 30, 2021 at 20:23

I've never quite seen NxN, but I think it may be referring to this scenario:

If N examples are available, N training sets (subsamples) with N-1 examples are created, such that each of those N training sets (subsamples) omits exactly one (qty. 1) example that (collectively) every other training set (subsamples) did not omit.

This strategy is a limiting case where training set sizes are maximized, but so are the number of iterations. Thus it would be N iterations, each with N-1 examples. (I suppose this would be: NxN-1).

That's my humble guess.

• In other words, your guess is that $N\times N$ cross-validation is $N$ repeats of leave-one-out cross-valiation?
– Sycorax
Nov 30, 2021 at 20:25
• Exactly. Well said. Nov 30, 2021 at 20:28

To be precise, most probably you are referring to N x K-fold cross-validation (CV), i.e. the two numbers are not necessarily the same. This is usually referred to as repeated k-fold CV.

The N in NxK-fold CV refers to how many times the K-fold CV is to be repeated, each time with different partitions of the dataset.

This does not exactly conform to your picture, which is not entirely accurate, since it implies that the folds consist of continuous chunks of data, which is not the case in general.

Here is the relevant excerpt from Applied Predictive Modeling:

Repeated k-fold CV does the same as above but more than once. For example, five repeats of 10-fold CV would give 50 total resamples that are averaged. Note this is not the same as 50-fold CV.