Are there heuristics/techniques that can be employed to guard against overfitting of a model that do not require/employ human inspection of the learned model's performance.

So for example, they would look at the performance of a predictor and come up with a score of how likely it is that the model has been overfit.