Does k-fold cross validation always imply k uniformly sized subsets?

Question

I'm a bit confused on a minor point that I'm trying to discern due to a cross-validation strategy I've come across in my work that creates k-folds but the folds are not of equal length (for example some folds are of size 17, another 18, on up to 24). Is k-folds cross validation constrained to folds of equal length? Arbitrary choices of training data length and fold count can yield fractional numbers of course where one fold will draw the short stick but would it be accurate to say k-fold attempts to make roughly equal fold sizes?

In particular I'm hearing contradictory messages from in this question

Matt Krause "divided into different, mutually-exclusive 'folds'"

a Data Head "k-fold cross-validation (kFCV) divides the N data points into k mutually exclusive subsets of equal size."

Answer 1

I didn't meant to imply that the folds should be different sizes in that answer (and I'll update it to match).

Each fold should contain an equal number of observations, or as close to equal as possible. If you want to perform 10-fold cross-validation on $N=101$ observations, one fold will have 11, rather than 10, items in it. That's fine. If you have 102 observations, it'd be best to have two folds of 11 items each, rather than 1 of 12 and 9 of 10, though I doubt this matters much in practice, particularly as $N/k$ increases.

There is no need to choose $k$ that divides evenly divides $N$ (how would you even run cross-validation on a prime-number-sized data set?), or discard examples until $N$ is evenly divisible by $k$ (data is hard to get; don't throw it away).

Answer 2

Just one caveat to keep an eye on - if the fold sizes are not equal then the average of fold results is not the same as the average over whole dataset. You should weight fold results by fold size to get the correct average.

Suppose you have a dataset $X=\{1,2,3,4,5\}$ and use 2-fold cross-validation. The first fold is $\{1,2\}$ and the second fold is $\{3,4,5\}$. Let's say your predictions for the first fold are $\{2,2\}$ and for the second fold $\{4,4,4\}$. Then MSE for the first fold is $(1 + 0)/2 = 1/2$ and for the second fold $(1 + 0 + 1)/3 = 2/3$. Naive cross-validation result would be $(1/2 + 2/3)/2 = 7/12$. But if you calculate the same thing over entire dataset you get $(1 + 0 + 1 + 0 + 1)/5 = 3/5$. The correct way to calculate cross-validation results would have been $1/2*2/5 + 2/3*3/5 = 3/5$,

Answer 3

$k$-fold cross-validation will divide your data into $k$ subsets. Then you iterate $k$ times: $i=1,2,...,k$, using the union of subsets except the $i$th subset to train your model. Then testing that model on the $i$th subset. If $k$ is not a divisor of the amount of observations you have in your data set it is impossible to get subsets of equal size. This can happen easily and you usually end up with subsets of approximately the same size, depending on how the splitting of data is implemented in the tool you are using. You can, for example, have $k-1$ subsets of the same size and one which compensantes by having a different amount of observations than the rest.