First of all, you need to choose before the final test. The purpose of the final test is to measure/estimate generalization error for the already chosen model.
If you choose again based on the test set, you either
- need to restrict yourself to not claim any generalization error. I.e. you can say that your optimization heuristic yielded model x, but you cannot give an estimate of generalization error for model x (you can only give your test set accuracy as training error since such a selection is part of training)
- or you need to get another test set that is independent of the whole training procedure including selecting between your two candidate models, and then measure generalization error for the final chosen model with this third test set.
Secondly, you need to make sure that the more overfit model actually outperforms the less overfit model in the test: Test set results do have random uncertainty, and this is known to be large for figures of merit like accuracy which are proportions of tested cases. This means that substantial numbers of tested cases are required to guide such a decision between two models based on accuracy.
In the example, a difference such an in the question can easily need several thousand test cases to be significant (depends on the actual distribution of correct/wrong predictions for both models, and on whether only those 2 models are compared).
Other figures of merit, in particular proper scoring rules, are much better suited to guide selection decisions. They also often have less random uncertainty than proportions.
If model 2 turns out not to be significantly better*, I'd recommend choosing the less complex/less overfit model 1.
Essentially this is also the heuristic behind the one-standard-deviation rule: when uncertain, choose the less complex model.
* Strictly speaking, significance only tells us the probability to observe at least such a difference iff there is really no difference in the performance [or if model 2 is really no better than model 1], while we'd like to decide based on the probability that model 2 is better than model 1 - which we cannot access without further information or assumptions about the pre-test probability of model 2 being better than model 1.
Nevertheless, accounting for this test set size uncertainty via significance is a big step into the right direction.