Despite there are multiple questions about it, I cannot figure a solution about my problem. I have built a simple neural network classifier on the MNIST database. I have divided it in training, validation, and test sets. Then on the first two sets I have obtained the hyperparameters, in particular the number of epochs following an Early Stopping procedure.

Then I train a new model with the same architecture of the previous one on traininig+validation for the same amounts of epochs. I do not understand why this new model performs poorer on the test test in comparison to the old one. I can also share the code, but I know that CrossValidated is for more conceptual questions.

  • $\begingroup$ How much worse? // Why shouldn’t it perform worse? // How much fitting to your validation set are you doing before you move to the test set? $\endgroup$
    – Dave
    Mar 29 at 11:28
  • $\begingroup$ It should perform better because in the second model the training occurs on more data. Reading some papers about the topics, the approach of retraining on the validation+training set is recommended. The performance difference is small for this database, less than 1%. The same gap is approximatively about 8-10% for a real image database that I use for work. $\endgroup$
    – Jonny_92
    Mar 29 at 12:00


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.