In addition to this question I want to know how the "outputs" in a leave-one-out CV have a higher variance.
I understand the answer provided that says that when elements of each sample are highly correlated, the variance of the means is higher (you could have a sample with one high valued observation, which in turn will make the others also high valued due to the correlation, which will give you a high mean and vice versa for low valued observations. This in turn gives you a high variance between the means).
Now this is the reason why in LOOCV we have a higher variance, because their outputs are highly correlated. What I now do not understand is how this plays out.
Let's say we have a sample of 20 where we do LOOCV on. We thus create 20 models (where each observation is the validation set once). There is a lot of overlap between the training data, the models will be a like and thus their outputs (high correlation). But where is this variance between the means than? As I see it there is only one mean, namely the mean of the 20 outputs provided by the 20 models?