If I test various models and the best performing model also happens to be one that appears to be overfit, is this an issue? For example, if I have a model with 100% accuracy on the training data and 75% on the test data and I compare it with a model that has 65% accuracy on both training and test, is it a problem to select the first model?
I guess you could argue that maybe my first model could be improved in some manner so that it is no longer overfitting and thus test accuracy would also likely improve, but what if it couldn't? Is the large discrepancy between training and test error a warning that future data may not predict well? An issue of stability perhaps?