How do I implement nested K-fold cross-validation when it comes to k-Nearest Neighbours?
Let's say I built a kNN classifier, and used K-Fold CV to tune the hyper-parameter. Now, how do I use nested K-Fold CV? I have read multiple articles, but they don't explain it well enough (esp. in the case of kNN).
From my understanding, in nested CV:
I do K-Fold CV with K = 5 and k = 1, for example, on the training data and see the mean error rate. Then I do CV again with K = 10, for example, and k = 1, and then I do it again, with K = 15, for example, and k = 1, and so on, for multiple values of K.
Then I repeat the whole thing for k = 2, and so on, for multiple values of k.
In the end, I can use the data to plot a graph to see the what the mean error rate is with multiple values of k and for multiple values of K. So the X axis = k values, y axis = mean error rate and I can plot K lines.
And so I can look for the value of k with the minimum mean error rate, for the biggest K I could find, and use that value with the classifier to test out-of-sample accuracy on the test set.
Is that what is meant by nested CV?