Nested cross validation from a practical perspective

Question

I'm very confused about nested cross validation works . I have seen that nested cross validation is necessary to split parameter selection and model selection, since using the test set to both select the values of the parameter and evaluate the model, I risk optimistically biasing my model evaluations. But it is not clear how exactly it works. I would like to confirm that my way of proceeding is correct, or that some people explain to me how to proceed.
Actually, I do so:

1. Split the traing set int 3 parts: training set, validation set and test set
2. I do a cross validation (5-fold-validation) on training and validation set.In this way for each parameter I have an extent that allows me choice the best parameters. Namely in this loop I find the best parameters(thus a model)
3. And now model selection. Alone for the best parameters found I have to measure the accuracy. I have to consider the training set, the test set and the validation set and do another 5-fold-validation to find a reliable measure.

Is it right? Furthemore, those loop in the literature are called the inner loop and outer loop and aren't nested, namely before I do the inner loop (the point 2 above) and later apart I do the outer loop i.e. a single cross validation on model with best parameters found in the inner loop. Is it right? How do you etablish the initial size of training set,test set and validation set?

Thanks in advance to who will help me. (I'm using svm, but this isn't very important)

Answer 1

No, it's wrong. The nested cross-validation do a k1-fold-validation splitting the data in a training set and a test set(a fold). Later for the training set it is make another k2-fold-validation (the inner loop). This inner loop is make for each parameter combination for find the best model.In the outer loop is done the same thing for the others splitting combinations of the data and in this way at the end k2 models are found. With the rispective test set it is measured the performance of each found model. These k2 measures are averaged. In this way you do performance evaluation of the method. The purpose of nested cross validation isn't find the best model (parameters) but a measure of the method(SVM,etc..)(not a measure of the model).

For do performance evaluationan and model selection too I have to do: Split the data in training set,validation set and test set. I have to do cross validation on the training set and validation set with for example grid search to find the best model. Measure this found model performances with the test set.