I am working in text classification in RapidMiner where, because of the nature of my problem, I cannot use the built-in k-fold cross validation strategy, so I decided to create 10 copies of my dataset manually in order to incorporate the k-fold cross validation $(k = 10)$. However I have a few confusions regarding it. I have a dataset of 4000 samples distributed in 20 categories.
I am thinking to incorporate 10-fold cross validation by creating 10 copies of same dataset where 90% (3600) will be used for training and 10% (400) for testing. In each of the 10 copies of dataset, the test set will be different from other datasets. In the end of the classification task I will average their results. Am I right about my interpretation of 10-fold cross validation?
When we say to divide data in k-folds, use k-1 folds for training, and use the last one for testing, do we mean to train the classifier for k-1 times on k-1 folds, or do we mean to train the classifier only once on k-1 folds?
E.g. we have a data set of 4000 samples which is divided in 10 folds with 400 samples in each fold. By training the classifier on k-1 folds, do we mean train the classifier k-1 times (here 9) on k-1 folds (9) with 400 samples in each fold, or do we simply mean to train the classifier once on (400 × 9 samples) and test it on the last fold?