# Neural network training without early stopping

I was researching k-fold cross-validation, and read that one should train on k-1 of the k partitions and test on the remaining partition, and then repeat for each partition, averaging the results to get an estimate of model performance. This I understand; however, if there is no validation dataset, when should I stop the training (since early stopping is not possible, and so the model cannot simply be trained until generalisation performance starts to worsen)? I.e. should training be stopped after a set number of epochs, or when the gradient falls below a certain limit? Are there any tips for what these stopping parameters should be?

It’s actually very simple. Just use $k-2$ folds for training, 1 for validation and 1 for testing. You probably never encountered this issue before, because fitting most of the “classical” statistical learning methods(SVMs, OLS, PLS, splines, GAMs, Gaussian Processes, etc.) corresponds to solving a convex optimization method: there is one and only one solution, and approximating it more or less accurately is “just” an issue of numerical analysis. Also, these methods don't have an overwhelming capacity, such as Deep Neural Networks do, and you don't use early stopping (as one of tools) to control overfitting. This is why you never had to use a training/validation/test split when doing cross-validation before.