Context: I'm constructing a CNN classifier for text categorization. I have a dataset with 20 different classes and approximately 20,000 labeled features (the 20 News Group dataset for those interested).

I'm wondering if I'm training my model on too many epochs, which would make it really good at categorizing the features from my training dataset, but unable to adapt to new / slightly different inputs. Is that what we call "overfitting"? The term is not clear to me.

Also I would like to clarify the term "convergence" of a neural network. Is this convergence attained when the accuracy starts plateauing? Or is it related to the loss value?


2 Answers 2


Pankaj Daga's expansion is great, I'll take care of the illustration. Here is a typical curve when training a neural networks:

enter image description here

The reported F1-score for the test set should to be the F1-score of the test set of the epoch where the F1-score of the validation set was the highest. (this is called "test best" on the figure)

  • $\begingroup$ Thanks, that graph complement perfectly the first answer :) $\endgroup$ Aug 19, 2016 at 16:57

Your comment regarding the epoch is true. So, if you use too few epoches, you may underfit and using too many epoches can result in overfitting. As you know you can always increase the training accuracy arbitrarily by increasing model complexity and increasing the number of epoch steps. One way to try and alleviate this problem could be through early stopping. In pseudocode:

  • Split data into training, validation and test sets.
  • At every epoch or every N epoch:
    • evaluate network error on validation dataset.
    • if the validation error is lower than previous best, save network to epoch.
  • The final model is the one with the best performance on validation set.

This is very similar to the classical cross validation techniques you use in machine learning approaches.

Regarding convergence, you usually say the network has converged to some local minima if your error metric and weights are relatively constant over several iterations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.