Usually we partition the whole data-set into three parts: training data, validation data and test data. For the training data it is used to train the parameters of the variables in the model; and the validation set is for choosing or check the overfitting of the parameters; and for the test set it is only for testing the performance of a particular set of parameters. That's the measure of the generalization ability of our model.
And I also learned that if the data-set is too small, it is better to use all of the data as training data. But in such case how do we validate or test the model?
What are confusing me include:
1) When we divide our data-set into the three parts how can we do it in a right way in different scenarios? For instance if the data-set is too small it would be 1:0:0; but how small it should be to do that way? What if the size of our data is very large?
2) Should we just do the partition or we can process then separately afterwards? For instance if the test set is relatively small we can optimize those data. Do some data cleaning futher or something, which makes it different from the training data but more approximating to the real scenario if we try to apply the model.
3) I know that there is a very popular method named 10-fold crossvalidation, and it is also often utilized when we have small data-set. When to choose which method?