1
$\begingroup$

I'm quite new to the field and need your advice. I'm training an artificial neural network on a very small dataset (~30,000 samples). I have difficulties judging if my model is overfitting or not.

Based on the output from keras.evaluate() everything is fine. Accuracy for training is ~98% while testing is ~96%. But if I look at the history it paints a different picture. There you can clearly see that testing accuracy is above the training accuracy, which would be an indicator for overfitting, right? On the other hand, the loss function is fine. enter image description here

So what is more relevant? The results from the keras.evaluate() or the history plots? Or is it a combination of both? And if you agree that my model is overfitting, is this going to be an issue? All of this will be part of a publication, and as I'm new to the field, I want to make sure that the model holds up to the reviewers.

If it helps, the models' architecture is: dense(100) dropout(0.5) dense(50) dropout(0.5) dense --> Output

$\endgroup$
8
  • $\begingroup$ I have added the labels. $\endgroup$
    – TheoBoveri
    Oct 19 '21 at 20:28
  • 1
    $\begingroup$ This looks like a plotting issue to me. Are you sure you labelled train/test correctly in the plot? Bugs like this can easily happen. But before you start acting on it, be sure it's not the code. $\endgroup$ Oct 19 '21 at 22:16
  • $\begingroup$ Also, I wouldn't call this overfitting. Train and test error align well after a few epochs. Seems like your model actually learned something useful from the train set and was able to generalize well. $\endgroup$ Oct 19 '21 at 22:22
  • $\begingroup$ @LaksanNathan this was also my first thought, that something went wrong with the labels. But that's actually not the case. $\endgroup$
    – TheoBoveri
    Oct 20 '21 at 6:12
  • $\begingroup$ @LaksanNathan so you would say, that, as long as the curves more or less align, I shouldn't worry about overfitting? I was a bit concerned about the intersection of the loss curves at the 50 epochs mark, but was hoping, that the 'overfitting' was so small, that I acutually would be that serious. Thanks a lot for the reassurance! :) $\endgroup$
    – TheoBoveri
    Oct 20 '21 at 6:21
0
$\begingroup$

Note that there can be the effect called double decent, where additional training still helps your performance on the test set, although your training error is already well below the test error:

https://openai.com/blog/deep-double-descent/

$\endgroup$
1
  • $\begingroup$ Thanks for the advice with the double descent, I'll look into it! The only thing is, that I'm actually not that concerned about my test set performance. I'm completely fine with the current ~96%. What I'm concerned about is the apparent overfitting and if I am reading the graphs right or wrong or if I'm overvaluing the topic of overfitting in my case. $\endgroup$
    – TheoBoveri
    Oct 19 '21 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.