Assigning even partitions for Cross-Validation This is a very basic question about cross-validation. Say that I have a sample size of 2901(or any difficult to divide number). How do I split this up into equal partitions (other than n=1)? And how big should I make each partition?
For example, if I make each partition size 300 (which gives me approximately 10 partitions), I will have some data points that are in more than one partition, giving it an unfair weight. Is this acceptable/what do people normally do about this?
By the way, I wanted to split it into equal partitions so that I can easily write code that will perform cross validation for any number of partitions.
 A: You could also draw a sample of size 2901 from a uniform value taking its values in {1, 2, ... 10} (for 10-CV). This sample determines a random partition of the data, with 10 sub-groups of roughly equal size (each sub-group has equal probability of being drawn for each training observation). 
For each of these sub-groups you can then compute a sub-group average measure (such as MSE, as opposed to total sub-group squared error) as suggested by Matthew.
I wouldn't think small differences in subsamble sizes would matter much if you take sub-group averages, but if you are worried you can always repeat this sampling procedure multiple times to get an average 10-CV estimate.
A: A lot of validation measures are usually 'averaged' over the whole partition to allow for direct comparison on other data sets. For example, mean square error for continuous prediction is
$$
\text{MSE} = \frac{1}{N} \sum_{i=1}^N (\text{prediction}_i - \text{actual}_i)^2.
$$
Other examples are MAE, area under the ROC curve, Brier score, R squared (generalized or otherwise). In this way, if the difference in size between partitions is small (say, within a few) then you shouldn't be worried about any kind of imbalance. In your case for 10-fold CV, (after shuffling the data to ensure random assignment) I would take $\lfloor 2901/10 \rfloor = 290$-sized partitions and give the last data point to any of the partitions. It won't matter which.
You can always make your validation measure size-independent by dividing by the partition size. For example, if you calculate logarithmic scoring in total, then just divide the total score by the partition size.
