I read over and over that the "Leave-one-out" cross-validation has high variance due to the large overlap of the training folds. However I do not understand why that is: Shouldn't the performance of the cross-validation be very stable (low variance) exactly because the training sets are almost identical? Or am I having a wrong understanding of the concept of "variance" altogether?
I also do not fully understand how LOO can be unbiased, but have a high variance? If the LOO estimate is equal to the true estimator value in expectancy - how can it then have high variance?
Note: I know that there is a similar question here: Why is leave-one-out cross-validation (LOOCV) variance about the mean estimate for error high? However the person who has answered says later in the comments that despite the upvotes he has realized that his answer is wrong.