What is the role of k-fold cross validation?

Question

When I search for the answer online, it seems like there is a disagreement on what cross-validation is. Some say k-fold cross validation is used to get an estimate of how well a model will perform before building the model on all of the data. See below:

How to choose a predictive model after k-fold cross-validation?

Some say k-fold cross validation is used on the training set of data to obtain an optimal model before applying the model to the testing set. If this way of k-fold cross validation is correct, how is the optimal instance of the model picked? k-fold cross validation will create k instances of the model, so what would be the way to pick one to use on the testing set to get an accuracy measurement.

Thanks in advance.

Answer 1

I think the accepted answer in your link provides highly valuable insights. I'd just like to point out the two uses of CV, which you seem to think as a disagreement in the community (despite that it's not):

Some say k-fold cross validation is used to get an estimate of how well a model will perform before building the model on all of the data.

This is just model checking, and is applied when you've decided on your model, and want to have a statistically better estimate of what you'd achieve with this model, compared to using only one test set.

Some say k-fold cross validation is used on the training set of data to obtain an optimal model before applying the model to the testing set.

This is model selection. You can choose the best model or hyper-parameter set via this procedure. But, note that, when you use your data for this purpose, i.e. exploited it to see if which configurations best fit your data, and later if you use the above procedure to check the test performance of your model, the performance will be optimistic since you've tuned your model(s) with the data you aim to test on.

Answer 2

Since the test set is not available at the time of model fitting, only the training set, cross-validation can only be done on the training set which it splits into smaller training sets as well as a pseudo-test set called the validation set. Your confusion has only to do with which dataset (train/test) is available at the time of model fitting. The train set we have, but not the test set. The test set is actually the future. Remember we cannot see the future.