A very simple question: What is cross-validation for?
As far as I understand, cross-validation is used for selecting the model and not the parameters of the model, but I want to check if I am right. The famous k-fold, illustrated in this image:
uses $k$ combinations of train-test samples to train and test the model and is used to avoid overfitting.
So if we have a model M, Is the model trained from scratch for every train-test combination?
So are these steps correct?
repeat k times: train M with sample train[i] predict test[i] with M compute MeanSquaredError[i] for test[i] i = i+1 end repeat compute mean of MeanmumSquaredError
As the model is re-trained everytime this is only usefull to check if the model is well chosen and not the parameters of the model, right?
UPDATE: Let's suppose the model M is a neural network with one hidden layer. Do you use cross-validation for example to select the number of neurons in the hidden layer?