calculating recall/precision from k-fold cross validation (or leave-one-out) can be performed either by averaging the recall/precision values obtained from different k folds or by combining the predictions and then calculate one value for each of the recall and precision.
My data is composed of 1000 samples belonging to two classes. the positives are only 50 while negatives are 950 samples. If we apply leave-one-out using the averaged k-fold cross validation approach. Then, we will notice that we have the precision and recall in 950 folds are not defined (NaN) because NP/(TP+FP) are all zeros (as only one negative sample is examined in this k-fold). I think, such NaN might be replaced by 1 because no meaning to set its value to be 0 (because the current sample is negative and it was correctly predicted to be negative). Then, we average the recall/precision from 1000-folds. the values will be different than obtaining the all the predictions and then calculating one precision/recall.
The same difference will happen when we apply k-fold cross validation on this data instead of leave-one-out (even if we produce balanced ratio of positives in each fold).
So, my question is: which approach is more accurate to calculate the precision/recall from k-fold cross validation (combined or average)? is the same situation for leave-one-out?
There are two relevant discussions : Mean(scores) vs Score(concatenation) in cross validation and Averaging precision and recall when using cross validation
However, the first one did not have clear answer for this point and the second discusses the averaging effect of Fmeasure (not in the precision/recall). both discussions did not discuss if leave-one-out has other preference.