When you fit() a model, in let's say Keras, over a large number of epochs, chances are overfitting will occur. When supplied with a validation-set, you can easily find the point where the validation was best. (This can be done automatically if you make checkpoints)

However, this "best solution", is not returned. Instead, the last tested solution is returned, which will be a very overfitted one if the number of epochs is large.

This is strange to me, but ok... I'll deal with it and just reload the best solution.

However... when using K-fold CV, it will always calculate the average score out of the scores of the last solutions of every iteration. Why is this? Doesn't this make K-fold CV useless? Is there something I'm missing? (Or doing wrong?)

An example of where this happens: I have an LSTM fitting a graph. K-fold with 100 iterations give me a variance score of 0.22. K-fold with 500 iterations gives me a score of... 0.05... This does not make sense to me, because it will have passed the 100 iterations point as well, when doing 500.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.