From only the first epoch of training my NN with SGD (I use Xavier initialisation for weights), the accuracy shoots off to 92%, and then flattens out. The same thing happens with loss (but lower, of course). Is this a sign of overfitting, and if so: what can I do to decrease it?

  • $\begingroup$ no, high test set accuracy is not an indication of overfitting $\endgroup$ – shimao Dec 5 '19 at 22:18
  • $\begingroup$ Is your dataset imbalanced? How do the actual predictions look like? $\endgroup$ – Jan Kukacka Dec 6 '19 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.