0
$\begingroup$

enter image description here

I am writing a network that classifies different species of butterfly, I have 9 epochs total. I have reached a wall as my major is in Physics, I am wondering if anyone can spot any distinct issues with these results - I feel like this may be a case of over fitting but I am not sure.

While doing some reading I understood that "Overfitting if: training loss << validation loss" To what degree is the true?

Improve this question
$\endgroup$
4
  • $\begingroup$ What is your definition of overfitting? Do you think these images fit that definition? Why or why not? $\endgroup$
    – Sycorax ♦
    Nov 21, 2019 at 18:51
  • $\begingroup$ Over fitting can manifest itself as a significant difference between validation and training accuracies. Because training and testing accuracies are similar for your case, I dont think you have a problem of over fitting. That being said, I dont understand what you mean by wall. If you are not happy with the accuracies, you may increase the complexity of your model(e.g more layers, hidden units and so on). Also, please confirm that you dont have imbalanced data problem. $\endgroup$
    – prony
    Nov 21, 2019 at 18:53
  • $\begingroup$ @ReinstateMonica While doing some reading I understood that "Overfitting if: training loss << validation loss" $\endgroup$
    – Tino Uchiha
    Nov 22, 2019 at 12:00
  • $\begingroup$ @prony yeah no imbalanced data, my main thinking that it is overfitted is to do with the above comment. $\endgroup$
    – Tino Uchiha
    Nov 22, 2019 at 12:01

3 Answers 3

Reset to default
0
$\begingroup$

Generally your test scores will be to some extent worse than your validation scores, but large differences can be signs of overfitting. Do you consider the difference between your scores to be large? What are your needs in terms of performance?

I'd imagine most people would not diagnose this as overfitting. But it's important to mention overfitting isn't an absolute state and will depend on your needs.

If you're dissatisfied with your validation performance, you can see there's not a tremendous amount of room to improve against your test performance (thus, overfitting is unlikely and/or won't net you great results). You might be better served by looking at alternate model representations or model parameters, rather than trying to tackle overfitting.

Improve this answer
$\endgroup$
3
  • $\begingroup$ While doing some reading I understood that "Overfitting if: training loss << validation loss" what do you think? $\endgroup$
    – Tino Uchiha
    Nov 22, 2019 at 11:59
  • $\begingroup$ It's a useful rule of thumb. It's better to understand what overfitting means. If your model is overfit, it has memorized the wrong patterns to increase its performance on your training set. As a result, when you point it to unseen data, it will generally not perform as well because it's looking at the wrong things. Thus, training loss << validation loss. There are many resources to help you with understanding overfitting. $\endgroup$
    – Julian Drago
    Nov 22, 2019 at 19:25
  • $\begingroup$ Also, a nice illustration of the idea: xkcd.com/1122 $\endgroup$
    – Julian Drago
    Nov 22, 2019 at 19:28
0
$\begingroup$

As @Julian Drago wrote it does not seem that you are having overfitting. To have a better glance you can plot your predicted values against your measured values and color your points according to the subset they belong to (train and validation). If the model is overfit the training point would be much closer to the 45 degrees line.

I attach an example from one of my projects, here the model is not overfit. what it is indicated as test is corresponding to your validation set enter image description here

Improve this answer
$\endgroup$
0
$\begingroup$

Generally overfitting means that you give your fit too many degrees of freedom (by trying too many things out or optimising over a too big space), as a result of which the fit won't generalise well and the training loss of your chosen/"optimal" fit will be much lower than the validation loss (although occasionally and accidentally you may find a good generalisable fit when doing overfitting).

Overfitting is not a binary yes/no thing. What happens in your images is that if you optimise training loss/accuracy you find a fit that has a validation loss/accuracy that is worse (than training) and not optimal (compared to other validation results). So with stopping earlier, i.e., "trying out less" - not sure what the numbers on your x-axis actually mean - you could've found something that is better regarding validation loss. So trying out more has "masked" something that would've been better. That's overfitting.

The story is not quite clear cut here though, because the actual optima of the training loss/accuracy seem to appear at points where the validation loss/accuracy isn't that bad either (if I see things correctly), so there is some overfitting but it may not hit that badly here and may be tolerable. As I said, it's not always black or white.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.