I'm doing a two-class image segmentation problem using fully convolutional networks, and got the following loss curves and learning_rate change curves. From the loss curves, it seems the network is overfitting from the training set. I have used the weight decay and changing learning rate policy. Are there any other methods to preventing the overfitting problem?

The loss curves for training data and validation data are (blue line is loss curve on validation data):

loss curves

The learning_rate policy is piece-wise changing. It seems lowering the learning rate cannot change the loss values a lot.

learning_rate curves


1 Answer 1


Here are two ideas:

  • Add random noise to the training set images. This should make the training set harder and improve the generalization error.
  • Early stopping. I suggest this because your error on the validation set has a "U" shape. For this you would need to store a copy of the parameters every time the error on validation data improves. The training could stop when the error does not decrease for a given number of epochs, and at the end of the training you keep only the set of parameters that achieved the best error on the validation set.
  • $\begingroup$ Hi, thanks a lot for your kind answer! Yeah, I have tried this. $\endgroup$
    – mining
    Commented Dec 18, 2017 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.