Say you want to tune several parameters of your model using $N$ data. What you usually do is splitting your $N$ data into 3 sets:
- learning set: used to build your model;
- validation set: used to select the best parameters of your model;
- test set: used to test your model with the parameters selected on the validation set.
If you are using a CV procedure, the learning set and validation set are in fact the same and thus you only have a learning set + a test set. But my problem is that a cross-validation procedure does not avoid the appearance of overfitting. Thus, you could keep the 3 sets even if one uses CV. In that way, you use CV on the learning set to learn the parameters, you determine when to stop the learning by looking at the error on the validation set and then you test your parameters on the test set. Is that correct?