You need to stop training when the generalisation error, the error your model will make on unseen data, increases. Since you cannot measure this error directly by definition, one typically uses an approximation by evaluating the model on a held out (validation) set. When the error in this set increases, one stops. Cross-validation can also be used, but the variance of the error estimator will be higher (meaning: more noisy values) and you will have to use some sort of smoothing or heuristic to decide when to actually stop.

On the other hand for certain classes of regression models in reproducing kernel Hilbert spaces there exist bounds related to the number of samples, indicating when early stopping should occur.

You need to stop training when the generalisation error, the error your model will make on unseen data, increases. Since you cannot measure this error directly by definition, one typically uses an approximation by evaluating the model on a held out (validation) set. When the error in this set increases, one stops. Cross-validation can also be used, but the variance of the error estimator will be higher (meaning: more noisy values) and you will have to use some sort of smoothing or heuristic to decide when to actually stop.

On the other hand, for certain classes of regression models in reproducing kernel Hilbert spaces there exist bounds related to the number of samples which provide early stopping rules.