I am not sure that there are any systematic way to achieve residual benchmarking across all model types. Any model domain calls for different testing and what have you.
I just want to clarify a few concepts and share some others that are useful within the domain of modeling macroeconomics time series (econometrics modeling).
First, clarification... a model that has larger out-of-sample error vs. in-sample error is not an overfit model. This is by definition. Unless by pure fortuitous randomness, a model typically will make smaller errors on the learning sample where it fully captures the data info vs. the out-of-sample where it does not capture any info. The only typical situation is when the out-of-sample period is a lot more stable than the in-sample one. Let's say your in-sample period is a monthly time series between 2005 and 2010, around the Financial Crisis. And, your out-of-sample period is from 2013 to 2016; it is not unlikely your residuals will be lower during the out-of-sample period. But, you have to understand why. It is not necessarily because your model is great. It is in good part, because the in-sample period was so much more volatile.
And, that is the challenge with residual analysis. Although, it is the best and most extensive tool there is to test a model... you have to pay close attention whether the relative out-of-sample model performance is a function of the relative volatility difference in the data or due to the model structural strength or weakness. And, only studying the data and the model with a real critical eye can discern the answer.
When comparing the in-sample vs. out-of-sample residual performance, there are a few metrics you can look at. One is the standard error of the model over those respective period. The latter does not work well when the size of your samples are very different (typically a lot smaller within the out-of-sample sample). Instead, you could just use the standard deviation of the residuals which is the same thing but not adjusted or in this case distorted by the different degrees of freedom within the in-sample larger sample vs. the smaller out-of-sample sample. There are other measures catered for this type of exercise including the Mean Absolute Error (MAE) and the Mean Absolute Percentage Error (MAPE). Depending on what your dependent variable is either the MAE or MAPE may be more relevant. And, you compare those between the in-sample and out-of-sample results.
This type of analytical metrics (residual standard deviation, MAE, MAPE) is typically more useful than any information criterion (AIC, BIC, etc.) that are at times difficult to interpret.
Also, the mentioned analysis (comparing MAE, etc.) is a lot more useful when not only comparing one model's in-sample vs. out-of-sample performance, but, when comparing two or more competing models. And, the MAE on the out-of-sample can provide you a lot of info in terms of selecting your best model.
In addition to measuring and comparing MAEs, I would also compare MEs or MPEs. That's because when you look at absolute errors you can mask the upward or downward bias of a model during the out-of-sample period. We know a model should have a mean error of zero over the in-sample period. But, we also know it won't be the case during the out-of-sample period. And, it is important to measure that bias in either direction.
So, your best model is typically the one that will have the lowest MAE, MAPE and also lower ME or MPE (smaller bias). Notice that this best model is not derived by simply looking at the model with the highest R Square and lowest RMSE within the in-sample period. Many get obfuscated by models with very high R Square on the learning sample without even testing the model appropriately on a series of out-of-sample tests.