The Wikipedia article on cross validation states
[...] cross-validation can be misused [...] By allowing some of the training data to also be included in the test set – this can happen due to "twinning" in the data set, whereby some exactly identical or nearly identical samples are present in the data set. Note that to some extent twinning always takes place even in perfectly independent training and validation samples. This is because some of the training sample observations will have nearly identical values of predictors as validation sample observations. And some of these will correlate with a target at better than chance levels in the same direction in both training and validation when they are actually driven by confounded predictors with poor external validity.
While I do not completely understand this paragraph (e.g. what is "better than chance levels"?), it makes sense to me that samples with "similar" input values should be all in the test set or all in the training set when doing cross-validation. Are there any published results on (1) how the groups for $k$-fold cross-validation should be chosen, and (2) how the cross-validated estimate may get worse when $k$ increases?