validation vs test vs training accuracy, which one to compare for claiming model overfitting?

Question

I have read on the several answers here and on the internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting.

But I am confused that which two accuracies/errors amoung test/training/validation should I compare to be able to see if the model is overfitting or not?

For example:

I divide my data for 70% training and 30% test.

When I get to run 10 fold cross-validation, I get 10 accuracies that I can take the average/mean of. should I call this mean as validation accuracy?

Afterward, I test the model on 30% test data and get Test Accuracy.

In this case, what will be training accuracy? and which two accuracies I compare to see if the model is overfitting or not?

This is my first question on this platform so please ignore errors.

Answer 1

Yes, you take the average of the ten. For models with normally distributed residuals, it is the mean squared residual you have to average.

Both the crossvalidation error and the test set error are unbiased. The corssvalidation error will be more accurate and is therefore prefered unless you have some practical reason for using the training/test set approach.

A big different between the cross-validation error and the error you get by fitting the model to the entire data set is a sign of overfitting. Similarly, a big difference between the test set error and the training set error.