No, Cross-validation is not just varying the size of training and validation sets. There is a reason for doing so.
It is basically two concepts of Machine Learning,
1) Our algorithm should perform well on "future unseen" data points - that means the data points, which are not in hand (Data which will be affected by future situations), so neither training data nor test data.
e.g Product reviews can change over time, maybe due to reasons like 1) product has been modified in the future or competitor's products are not received in good condition due to changed packaging material, and now the reviews will be little different from what (they used to be) we have in hand, so "unseen future data" may not be "exactly" same as our train or test data. Even in this situation, our algorithm should work well.
2) When we determine K we are using test data and that is somewhat like doing reverse engineering, so we may get 96% accuracy on Test data using k = 6, but it may not work for future unseen data. Here we've already seen(used) the test data while determining K.
If we divide data in cross-validation and on the basis of it we determine K, then we haven't used test data(test data is still unseen) and so the K will be more relevant when it comes to future unseen data.
we divide data normally in 3 parts - 60% training data, 20% cross-validation data, 20% training data.
Problem : Of the total n data points, we are using only 60% of data to compute nearest neighbors. Is there a way to use 80% of the data (train 60% + cross-validation 20%)? As most of the time, more the training data-> better is the algorithm. K-fold cross-validation is a solution here.
Randomly break train data into 4 parts: D1=20% of data, D2=20% of data, D3= 20% of data, D4 = 20% of data.
Now for different values of K, use combinations of 3 parts as training data and the one which is left out is a cross-validation data.
**1)** D1,D2,D3 <- Train & D4 <- CV
**2)** D1,D2,D4 <- Train & D3 <- CV
**3)** D1,D3,D4 <- train & D2 <- CV
**4)**D2,D3,D4 <- Train & D1 <- CV
This way we can use every part of training data for cross-validation and for training, so we are not wasting data.
Limitation: - Time it takes to compute the optimal/best K in KNN increases by K times if we use K-fold cross-validation.