Here is a question:

Is it true that: A classifier that attains 100% training accuracy and 70% test accuracy is better than a classifier that attains 70% training accuracy and 75% test accuracy


Probably not? 100% training accuracy is often a sign of overfitting - you've trained the model to perfectly identify your training data, which means it is likely to perform poorly on data that doesn't resemble that which it was trained on, which is shown in the fact that it doesn't do quite so well on the test data.

  • 1
    $\begingroup$ How about the other case where the training data performs worse than the validation set? $\endgroup$ – mathpadawan Oct 29 '19 at 3:16
  • $\begingroup$ The training error is uninteresting. You can always bring it up to 100%. Models should only be evaluated and compared using hold-out sets. In both cases, the classifier with better performance on test is to be preferred. $\endgroup$ – Laksan Nathan Oct 29 '19 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.