Say, we test an arbitrary regression or classification procedure on $n$ independent samples with leave-one-out cross-validation. This results in an estimate of the prediction error $e_n$ for each sample $n$.
Can these $e_n$ be assumed to be independent draws of a (probably unknown) distribution?
My intuition says no, because (1) the training set is almost the same for each test sample, and (2) samples are used for both, training and testing.
If my intuition is wrong, and errors are independent, what about k-fold cross-validation, where the same training set is used for groups of $n/k$ samples?
Disclaimer: I tried to ask this question as concisely and generally as possible. If it lacks detail or specifity, please comment and I will update the question accordingly.