accuracy decreases with number of folds in x-validation I am running a Sequential model in Tensorflow for binary classification. I cross-validate it using sklearn's KFold with 50 folds.
The strange thing is that the binary accuracy has a trend of decreasing as the ordinal number of folds goes up, as seen in the plot below. The binary cross-entropy fluctuates without such a trend.
My understanding is that everything in each fold is independent of the others, and that the training doesn't remember previous folds as it is rebuilt inside each fold. So I am surprised to see this trend. Is there any explanation?
Edit: The sample consists of ~19,000 entries, randomly shuffled. The two classes make up 35% and 65% of the sample.
Edit: The model's architecture:
model = keras.Sequential([
        layers.Dense(units=200, activation='relu', input_shape=[len(f_features)]), 
        layers.Dense(units=400, activation='relu'),
        layers.Dropout(rate=0.3),  
        layers.Dense(units=400, activation='relu'),
        layers.Dropout(rate=0.3), 
        layers.Dense(units=50, activation='relu'),
        layers.Dense(units=1, activation='sigmoid') ])


Edit / update: I ran several tests with different numbers of folds. This "trend" showed up in a couple but not in most; I conclude that it was a random effect indeed.
 A: So I think I know enough about the question to compile all the comments to a single answer.
As far as I know, there is no known effect such as "as the numbers of fold increase CV scores decrease." You usually would observe @DikranMarsupial suggested, "would expect if anything the accuracy to go up as the size of the training set becomes larger." And yes, as you increase the number of folds, your training size increases because you are dividing your data into smaller pieces. But it is also possible there is a saturation point, meaning performance does not improve after some threshold even if you add more data. Also, I agree with @whuber; there is no evident trend in your plot. If you try my previous suggestion and take the mean of all the simulations and plot them, I believe you will see that curve is smooth and no evidence of a trend.
Edit:
Then it is certain your plot is by chance. Since your data is shuffled, the order of your folds does not mean anything. You can change the order you plot the results as you wish, but you shouldn't make any conclusions based on line plots. The only thing you need to consider is the distribution of your scores. Your scores seem like they have a mean around 0.84 and vary around above the 0.80 threshold, and this is the only conclusion you should make from here. The concept of "Trend" in stats. means: a long/short term movement in an ordered series, for instance, a time series. Your scores are not an ordered series, so the trend is irrelevant.
