1
$\begingroup$

What is people's method of early stopping? I run my validation data against the trained network and if the error is > the previous low error then I increment a counter. If the error < the the previous low error then I record the new low and reset the counter. When the counter hits 3 I stop training.

I also continue training if the validation error is better than the training error. Here's my code:

    NetworkTestResult oResult = testANetwork(voNetwork, oTestData, oTestResults);
    System.out.println("Validation Error: " + oResult.getError());

    // is the validation error greater than our lowest result?
    if (oResult.getError() > nLowValidationError) {
        nIncreaseErrorCount++;
    } else {
        nIncreaseErrorCount = 0; // reset
        nLowValidationError = oResult.getError();
    }

    // if validation error is better than training error then carry on training
    if (oResult.getError() < vnTrainError) {
        return true;                
    }

    // if we've seen higher error 3 times then stop training
    if (nIncreaseErrorCount == 3) {
        System.out.println("Early stopping");
        return false;
    }

    return true;

Is this a good method?

$\endgroup$
1
  • $\begingroup$ Well, in this case you have kinda high risk of being stuck in a local minimum, I guess. $\endgroup$ Mar 5, 2018 at 15:32

1 Answer 1

2
$\begingroup$

There are multiple early stopping strategies. I recommend reading the article Early stopping - but when? by Lutz Prechelt (1997), where many stopping policies are described and evaluated. The one you proposed is often used, even though higher number than three is used.

What is often done in neural network training, however, is that you train for a fixed number of epochs (sufficient for the training error to converge) while recording validation error and saving weight snapshots every few epochs. At the end, you just select the snapshot with lowest validation error.

Also, note that there are many threads on this site describing early stopping.

$\endgroup$
2
  • $\begingroup$ I'd not actually thought of that; so I could store the NNs together with the train and validation errors, then train for large number of epochs and retrieve the optimal performer once I've done over a large number of epochs. $\endgroup$
    – BigBadMe
    Mar 5, 2018 at 16:55
  • 1
    $\begingroup$ Yes. All you need to store is the validation error and the network parameters. $\endgroup$ Mar 5, 2018 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.