The most direct way to assess overfitting is by looking at any performance differences on the training set compared to an independent test set. This can be automated easily.
In cross-validation, you can assess performance based on all training and test set pairs. Then at the end you can compare the results of training set predictions and test set predictions. If they are comparable, there was no overfitting. If the training performance was far better than test, there was overfitting.
Note that if you're going to perform model selection using cross-validation, you are unlikely to end up with a model that overfits, because that model wouldn't have good cross-validation performance anyway.