What to do next for preventing the overfitting?

Question

I'm doing a two-class image segmentation problem using fully convolutional networks, and got the following loss curves and learning_rate change curves. From the loss curves, it seems the network is overfitting from the training set. I have used the weight decay and changing learning rate policy. Are there any other methods to preventing the overfitting problem?

The loss curves for training data and validation data are (blue line is loss curve on validation data):

The learning_rate policy is piece-wise changing. It seems lowering the learning rate cannot change the loss values a lot.

Answer 1

Here are two ideas:

• Add random noise to the training set images. This should make the training set harder and improve the generalization error.
• Early stopping. I suggest this because your error on the validation set has a "U" shape. For this you would need to store a copy of the parameters every time the error on validation data improves. The training could stop when the error does not decrease for a given number of epochs, and at the end of the training you keep only the set of parameters that achieved the best error on the validation set.